1621696a6f log: restore `LogPrintLevel` messages from prior backports (Kittywhiskers Van Gogh)
52a1263989 merge bitcoin#25614: Severity-based logging, step 2 (Kittywhiskers Van Gogh)
21470fdeb3 merge bitcoin#25292: Add LogPrintLevel to lint-format-strings, drop LogPrint-vs-LogPrintf section in dev notes (Kittywhiskers Van Gogh)
026409e4ff merge bitcoin#25217: update lint-logs.py to detect LogPrintLevel, mention WalletLogPrintf (Kittywhiskers Van Gogh)
b046e091c9 merge bitcoin#25202: Use severity-based logging for leveldb/libevent messages, reverse LogPrintLevel order (Kittywhiskers Van Gogh)
7697b73257 revert dash#2794: Disable logging of libevent debug messages (Kittywhiskers Van Gogh)
ff6304f5f3 merge bitcoin#24757: add `DEBUG_LOCKCONTENTION` to `--enable-debug` and CI (Kittywhiskers Van Gogh)
88592f30a3 merge bitcoin#24464: Add severity level to logs (Kittywhiskers Van Gogh)
d3e837ad22 merge bitcoin#24830: Allow -proxy="" setting values (Kittywhiskers Van Gogh)
0e01d5b5f3 partial bitcoin#22766: Clarify and disable unused ArgsManager flags (Kittywhiskers Van Gogh)
a9cfbd1048 fix: don't use non-existent `PrintLockContention` in `SharedEnter` (Kittywhiskers Van Gogh)
f331cbe8c8 merge bitcoin#24770: Put lock logging behind DEBUG_LOCKCONTENTION preprocessor directive (Kittywhiskers Van Gogh)
d9cc2ea178 merge bitcoin#23104: Avoid breaking single log lines over multiple lines in the log file (Kittywhiskers Van Gogh)
479ae82ecc merge bitcoin#23235: Reduce unnecessary default logging (Kittywhiskers Van Gogh)
Pull request description:
## Additional Information
* This pull request's primary purpose is to restore `LogPrintLevel`s from backports in [dash#6333](https://github.com/dashpay/dash/pull/6333) that were changed to `LogPrint`s as they were backported before `LogPrintLevel` was backported.
* ~~`clang-format` suggestions for `LogPrintLevel` have to be ignored in order to prevent the linter from tripping due to a "missing newline" ([build](https://gitlab.com/dashpay/dash/-/jobs/8398818860#L54)).~~ Resolved by applying diff ([source](https://github.com/dashpay/dash/pull/6399#issuecomment-2488992710)).
* `SharedLock` was introduced in [dash#5961](https://github.com/dashpay/dash/pull/5961) and `PrintLockContention` was removed in [dash#6046](https://github.com/dashpay/dash/pull/6046) but the changes in the latter were not extended to the former. This has been corrected as part of this pull request.
## Breaking Changes
None expected.
## Checklist
- [x] I have performed a self-review of my own code
- [x] I have commented my code, particularly in hard-to-understand areas **(note: N/A)**
- [x] I have added or updated relevant unit/integration/functional/e2e tests
- [x] I have made corresponding changes to the documentation
- [x] I have assigned this pull request to a milestone _(for repository code-owners and collaborators only)_
Top commit has no ACKs.
Tree-SHA512: f2d0ef8ce5cb1091c714a2169e89deb33fa71ff174ce4e6147b3ad421f57a84183d2a9e76736c0b064b2cc70fb3f2e545c42b8562cf36fdce18c3fb61307c364
Some portions of the codebase make implicit assumptions that `GetQuorum`
will not return a `nullptr` by not performing checking for it or make
explicit assumptions by `assert`ing not-`nullptr`.
Let's document explicit assumptions, document implicit assumptions and
soften some hard assumptions where softening is possible.
It is impossible situation which will never happen.
But better to change it to exception for better error-prune implementation
in case someone will decide to change this code:
const auto quorums = qman.ScanQuorums(llmqType, pindexTip, quorums_to_scan);
if (bool isActive = std::any_of(quorums.begin(), quorums.end(), [&](const auto &q) { return q->qc->quorumHash == quorumHash; }); !isActive) {
return state.Invalid(TxValidationResult::TX_CONSENSUS, "bad-assetunlock-too-old-quorum");
}
...
const auto quorum = qman.GetQuorum(llmqType, quorumHash);
assert(quorum); <-- for sure exist because we just scanned quorums
7d9ff96091 merge bitcoin#16981: Improve runtime performance of --reindex (Kittywhiskers Van Gogh)
e531dff5f7 merge bitcoin#26417: fix intermittent failure in feature_index_prune.py (Kittywhiskers Van Gogh)
b04b71a957 merge bitcoin#24858: incorrect blk file size calculation during reindex results in recoverable blk file corruption (Kittywhiskers Van Gogh)
9e75b99c53 merge bitcoin#26215: Improve BaseIndex::BlockUntilSyncedToCurrentChain reliability (Kittywhiskers Van Gogh)
3bd584c845 merge bitcoin#24832: Verify the block filter hash when reading the filter from disk (Kittywhiskers Van Gogh)
e507a51323 fix: avoid `mandatory-script-verify-flag-failed` crash in bench test (Kittywhiskers Van Gogh)
a86109a017 merge bitcoin#25074: During sync, commit best block after indexing (Kittywhiskers Van Gogh)
e6867a35ce merge bitcoin#25123: Fix race condition in index prune test (Kittywhiskers Van Gogh)
baf6e26eed merge bitcoin#21726: Improve Indices on pruned nodes via prune blockers (Kittywhiskers Van Gogh)
c65ec190c5 merge bitcoin#24626: disallow reindex-chainstate when pruning (Kittywhiskers Van Gogh)
bcd24a25e3 fix: push activation height for forks ahead, fix `feature_pruning.py` (Kittywhiskers Van Gogh)
10203560f5 merge bitcoin#24812: Add CHECK_NONFATAL identity function and NONFATAL_UNREACHABLE macro (Kittywhiskers Van Gogh)
1caaa85716 merge bitcoin#24138: Commit MuHash and best block together for coinstatsindex (Kittywhiskers Van Gogh)
b218f123b7 merge bitcoin#23046: Add txindex migration test (Kittywhiskers Van Gogh)
ebae59eedf fix: make sure we flush our committed best block in no-upgrade cases (Kittywhiskers Van Gogh)
Pull request description:
## Additional Information
* When backporting [bitcoin#23046](https://github.com/bitcoin/bitcoin/pull/23046), it was discovered that there has been a longstanding bug in `CDeterministicMNManager::MigrateDBIfNeeded`(`2`)`()` that flagged a database taken from an older version for failing its "previous migration attempt", requiring the database to be fully rebuilt through a reindex.
This occurred because the older database would be read pre-DIP3 in `MigrateDBIfNeeded()`, which then caused the migration logic to write the new best block ([source](3f0c2ff324/src/evo/deterministicmns.cpp (L1236-L1241))) (the legacy best block is erased before the DIP3 condition is checked, [source](3f0c2ff324/src/evo/deterministicmns.cpp (L1233))) but while it completed the transaction ([source](3f0c2ff324/src/evo/deterministicmns.cpp (L1240))), critically, it didn't write it to disk (example of writing to disk, [here](3f0c2ff324/src/evo/deterministicmns.cpp (L1288-L1292))).
This meant that when it was read again by `MigrateDBIfNeeded2()`, it saw three things a) there is no new best block (because it didn't get written), b) there is no legacy best block (because it gets erased before the new best block is written) and c) that the chain height is greater than 1 (since this isn't a new datadir and the chain has already advanced), it concludes that it was a botched migration attempt and fails ([source](3f0c2ff324/src/evo/deterministicmns.cpp (L1337-L1343))).
This bug affects v19 to `develop` (`3f0c2ff3` as of this writing) and prevents `feature_txindex_compatibility.py` from working as expected as it would migrate legacy databases to newer versions to test txindex migration code and get stuck at unhappy EvoDB migration logic, to allow the test to function properly when testing against the latest version of the client, this bug has been fixed as part of this PR.
* In [bitcoin#23046](https://github.com/bitcoin/bitcoin/pull/23046), version v0.17 was used as the last version to support legacy txindex as the updated txindex format was introduced in [dash#4178](https://github.com/dashpay/dash/pull/4178) (i.e. after v0.17) and the version selected for having migration code in it (note, migration code was removed in [dash#6296](https://github.com/dashpay/dash/pull/6296), so far not included as part of any release) was v18.2.2 despite the range being v18.x to v21.x was a) due to the bug mentioned above affecting v19.x onwards and b) v18.2.2 being the latest release in the v18.x lifecycle.
* The specific version number used for v0.17 is `170003` as the binaries corresponding to `170000` are not populated in `releases/`, which causes a CI failure ([source](https://gitlab.com/dashpay/dash/-/jobs/8073041955#L380))
* As of `develop` (`3f0c2ff3` as of this writing), `feature_pruning.py` was broken due to changes in Core that were not adjusted for, namely:
* The enforcement of `MAX_STANDARD_TX_SIZE` ([source](3f0c2ff324/src/policy/policy.h (L23-L24))) from DIP1 onwards ([source](3f0c2ff324/src/validation.cpp (L299-L301))) resulting in `bad-txns-oversize` errors in blocks generated for the test as the transactions generated are ~9.5x larger than the now-enforced limit ([source](3f0c2ff324/test/functional/feature_pruning.py (L48C51-L48C57))), this is resolved by pushing the DIP1 activation height upwards to `2000` (the same activation height used for DIP3 and DIP8).
* Change in subsidy logic in v20 ([source](3f0c2ff324/src/validation.cpp (L1073-L1082))) that results in `bad-cb-amount` errors, this is resolved by pushing the v20 activation height upwards.
Additionally, an inopportune implicit post-`generate` sync ([source](3f0c2ff324/test/functional/feature_pruning.py (L215))) also causes the test to fail. Alongside the above, they have been resolved in this PR.
* As of `develop` (`3f0c2ff3` as of this writing), `bench_dash` crashes when running the `AssembleBlock` benchmark. The regression was traced back to [bitcoin#21840](https://github.com/bitcoin/bitcoin/pull/21840) (5d10b41) in [dash#6152](https://github.com/dashpay/dash/pull/6152) due to the switch to `P2SH_OP_TRUE`.
This has been resolved by reverting this particular change.
<details>
<summary>Pre-fix test failure:</summary>
```
$ ./src/bench/bench_dash
Warning, results might be unstable:
* CPU governor is '' but should be 'performance'
* Turbo is enabled, CPU frequency will fluctuate
Recommendations
* Use 'pyperf system tune' before benchmarking. See https://github.com/psf/pyperf
| ns/op | op/s | err% | ins/op | bra/op | miss% | total | benchmark
|--------------------:|--------------------:|--------:|----------------:|---------------:|--------:|----------:|:----------
| 17,647,657.00 | 56.66 | 0.1% | 231,718,349.00 | 42,246,265.00 | 0.1% | 0.20 | `AddrManAdd`
| 42,201,861.00 | 23.70 | 0.1% | 544,366,811.00 | 102,569,244.00 | 0.0% | 0.46 | `AddrManAddThenGood`
| 189,697.83 | 5,271.54 | 0.1% | 1,763,991.40 | 356,189.40 | 0.3% | 0.01 | `AddrManGetAddr`
| 454.38 | 2,200,808.04 | 0.6% | 6,229.11 | 1,343.92 | 0.1% | 0.01 | `AddrManSelect`
| 1,066,471.00 | 937.67 | 67.6% | 13,350,463.00 | 3,150,465.00 | 0.4% | 0.01 | 〰️ `AddrManSelectByNetwork` (Unstable with ~1.0 iters. Increase `minEpochIterations` to e.g. 10)
| 1,181,774.50 | 846.19 | 49.0% | 18,358,489.50 | 4,224,642.50 | 0.0% | 0.02 | 〰️ `AddrManSelectFromAlmostEmpty` (Unstable with ~1.1 iters. Increase `minEpochIterations` to e.g. 11)
bench_dash: bench/block_assemble.cpp:46: void AssembleBlock(benchmark::Bench &): Assertion `res.m_result_type == MempoolAcceptResult::ResultType::VALID' failed.
[1] 2343746 IOT instruction (core dumped) ./src/bench/bench_dash
```
</details>
## Breaking changes
None expected
## Checklist
- [x] I have performed a self-review of my own code
- [x] I have commented my code, particularly in hard-to-understand areas
- [x] I have added or updated relevant unit/integration/functional/e2e tests
- [x] I have made corresponding changes to the documentation **(note: N/A)**
- [x] I have assigned this pull request to a milestone _(for repository code-owners and collaborators only)_
ACKs for top commit:
UdjinM6:
utACK 7d9ff96091
PastaPastaPasta:
utACK 7d9ff96091
Tree-SHA512: e2f1e58abb0a0368c4f1d5e488520957e042e6207b7d2d68a15eb18662405a3cdac91c5ff8e93c8a94c0fdab9b1412bd608d055f196230506c1640439939c25d
d690309513 fix: limit amount of attempts for test `test_withdrawal_fork` (Konstantin Akimov)
ecd0a96f63 test: fix test_withdrawal_fork (UdjinM6)
6fbd128947 chore: clang format/reword/typos (UdjinM6)
5fd7b07ddc chore: test new validation of asset unlocks tests after fork 'withdrawals' (Konstantin Akimov)
27040030e9 fix: using proper quorums for asset unlock validation (Konstantin Akimov)
0253438503 feat: new fork WITHDRAWALS introduced (Konstantin Akimov)
Pull request description:
## Issue being fixed or feature implemented
https://github.com/dashpay/dash-issues/issues/83
## What was done?
Introduces new fork "withdrawals" which let Asset Unlock be valid for any active quorum + 1 extra inactive (in opposite of hard-coded 2 of them).
## How Has This Been Tested?
See new test section `test_withdrawals_fork` in functional test feature_asset_locks.py
## Breaking Changes
- new fork "withdrawals"
- new logic of validation of Asset Unlock transactions's signature. Even no asset-unlock is mined yet, previous version of clients will refuse blocks which is fine for current system
## Checklist:
- [x] I have performed a self-review of my own code
- [x] I have commented my code, particularly in hard-to-understand areas
- [x] I have added or updated relevant unit/integration/functional/e2e tests
- [x] I have made corresponding changes to the documentation
- [x] I have assigned this pull request to a milestone
ACKs for top commit:
UdjinM6:
utACK d690309513
PastaPastaPasta:
utACK d690309513
Tree-SHA512: cac28cf974e97d4a4100c22d5aa07b398b15413235383c68a0f4ff005005b28892c7ac8e6b703582c126cb6e0ff80a950f190dd32268655fac534ed6f2a90d03
This was spotted when working on `feature_txindex_compatibility.py`,
attempting to load old databases would fail because `MigrateDBIfNeeded()`
would delete `DB_OLD_BEST_BLOCK`, write `EVODB_BEST_BLOCK`, commit it
but never flush it.
So when `MigrateDBIfNeeded2()` would read it again, it would note the lack
of `DB_OLD_BEST_BLOCK` and conclude it was a failed run and exit. This
is solved by actually flushing our new best block, which would prevent
`MigrateDBIfNeeded2` from raising an objection.
It is not a breaking changes, because this fork is already happened in past
and no EHF transaction is in blockchain at that moment which requires versioning
It simplify implementation and unify RegTest, Mainnet and Testnet
No asset-unlock transaction has actually be mined yet, but v20 and mn_rr are activated long time ago.
So, this changes are not breaking changes
23812555b1 fix: possible deadlock during calculation of signals for historical blocks during re-index (Konstantin Akimov)
1087489fd4 feat: bury v20 deployment (Konstantin Akimov)
64cedb30bd feat: actually test something EHF unit tests (Konstantin Akimov)
762a808b8c chore: drop irrelevant bip9 code from feature_llmq_rotation.py (Konstantin Akimov)
7735631aad fix: remove v20 from test feature_llmq_evo as far as mn_rr used (Konstantin Akimov)
ca83b26815 fix: crash in CreditPool: it meant to check that DIP0003 is activated (Konstantin Akimov)
Pull request description:
## Issue being fixed or feature implemented
V20 is activated on mainnet: time to bury it!
https://github.com/dashpay/dash/issues/6186
## What was done?
Hard-fork v20 is buried and it requires to implement multiple fixes, simplifications, refactoring:
- some tests for EHF moved from functional tests to unit tests
- fixed crash in Credit Pool if DIP3 is not activated yet
- added a requirement for v20 activation for `CMNHFManager::GetSignalsStage`
- removed useless code from functional test feature_llmq_rotation
- renamed variables "v20" to "mn_rr" in feature_llmq_evo.py so far as actually used fork is mn_rr
## How Has This Been Tested?
Some unit and functional tests to succeed.
Done reindex (just in case):
src/qt/dash-qt -reindex -assumevalid=0
src/qt/dash-qt -reindex -assumevalid=0 -testnet
## Breaking Changes
N/A
## Checklist:
- [x] I have performed a self-review of my own code
- [x] I have commented my code, particularly in hard-to-understand areas
- [x] I have added or updated relevant unit/integration/functional/e2e tests
- [x] I have made corresponding changes to the documentation
- [x] I have assigned this pull request to a milestone
ACKs for top commit:
UdjinM6:
ACK 23812555b1
PastaPastaPasta:
utACK 23812555b1
Tree-SHA512: eec35745baa695f3f286d39b6a61fa0a9f34820b13d1dd4cfbd1efe707850283892c39bf7fe49c49c812e0c02465d64df11480b3f12aa7f21b59a71eeae7260e
9876c2d78b docs: add partial release notes (UdjinM6)
b330318db7 refactor: drop circular dependency (UdjinM6)
e54fe42ce8 refactor: use `key_to_p2pkh_script` in more places (UdjinM6)
3ed6246889 test: check `creditOutputs` format (UdjinM6)
ba0e64505b fix: `creditOutputs` in AssetLock tx json output should be an array of objects, not debug strings (UdjinM6)
Pull request description:
## Issue being fixed or feature implemented
Txout-s in `creditOutputs` for AssetLock txes should be shown the way txout-s are shown in other places. We should not be using debug strings there.
Example: `getrawtransaction 50757f651f335e22c5a810bd05c1e5aac0d95b132f6454e2a72683f88e3983f3 1`
develop:
```
"assetLockTx": {
"version": 1,
"creditOutputs": [
"CTxOut(nValue=0.01000000, scriptPubKey=76a914cdfca4ae1cf2333056659a2c)"
]
},
```
This PR:
```
"assetLockTx": {
"version": 1,
"creditOutputs": [
{
"value": 0.01000000,
"valueSat": 1000000,
"scriptPubKey": {
"asm": "OP_DUP OP_HASH160 cdfca4ae1cf2333056659a2c8dc656f36d228402 OP_EQUALVERIFY OP_CHECKSIG",
"hex": "76a914cdfca4ae1cf2333056659a2c8dc656f36d22840288ac",
"address": "yf6c2VSpWGXUgmjQSHRpfEcTPsbqN4oL4c",
"type": "pubkeyhash"
}
}
]
},
```
kudos to @coolaj86 for finding the issue
## What was done?
Change `CAssetLockPayload::ToJson()` output to be closer to [`TxToUniv()`](https://github.com/dashpay/dash/blob/develop/src/core_write.cpp#L262-L272)
NOTE: `refactor: use key_to_p2pkh_script in more places` commit is a bit unrelated but I decided to add it anyway to make it easier to follow assetlock creation vs getrawtransaction rpc check.
## How Has This Been Tested?
Try example above, run tests
## Breaking Changes
RPC output is different for AssetLock txes
## Checklist:
- [x] I have performed a self-review of my own code
- [ ] I have commented my code, particularly in hard-to-understand areas
- [ ] I have added or updated relevant unit/integration/functional/e2e tests
- [ ] I have made corresponding changes to the documentation
- [ ] I have assigned this pull request to a milestone
ACKs for top commit:
PastaPastaPasta:
utACK 9876c2d78b
Tree-SHA512: 158c98ac9e4979bb29c4f54cb1b71806f22aaec92218d92cd2b2e9b9f74df721563e7a6c5f517ea358ac74659fa79f51d1b683002a1cdceb1b8ee80f8fd79375
