5b4f34006dbd76223b55b156421f87d2241ac296 devtools, utxo-snapshot: Fix block height out of range (pablomartin4btc)
Pull request description:
<details>
<summary>Fixing a <a href="https://github.com/bitcoin/bitcoin/pull/28553#pullrequestreview-2251032570">bug</a> in <code>utxo_snapshot.sh</code>.</summary>
```
/contrib/devtools/utxo_snapshot.sh 840000 snapshot2.dat ./src/bitcoin-cli -datadir=${AU_DATADIR}
Do you want to disable network activity (setnetworkactive false) before running invalidateblock? (Y/n):
Disabling network activity
false
error code: -8
error message:
Block height out of range
```
And the user will see the following in the node and it would stay there if not reset:
```
2024-08-21T14:44:13Z UpdateTip: new best=00000000000000afa0cd000a16e244f56032735d41acd32ac00337aceb2a5240 height=235382 version=0x00000002 log2_work=69.987697 tx=17492185 date='2013-05-09T23:54:32Z' progress=0.016219 cache=71.0MiB(571085txo)
2024-08-21T14:44:13Z UpdateTip: new best=0000000000000087c5e0b820afff496b95ba44ad64640c73b234d3261d3f99d2 height=235383 version=0x00000002 log2_work=69.987750 tx=17492341 date='2013-05-09T23:54:47Z' progress=0.016219 cache=71.0MiB(571291txo)
2024-08-21T14:44:13Z UpdateTip: new best=000000000000014a4b5fddf3c8abb6209247255ca9e8df786b271dd1b2ac82a6 height=235384 version=0x00000002 log2_work=69.987804 tx=17492344 date='2013-05-10T00:20:18Z' progress=0.016219 cache=71.0MiB(571297txo)
2024-08-21T14:44:13Z SetNetworkActive: false
```
</details>
This is a "temporary" fix until #29553 gets merged, which will remove the script entirely.
Handle the "Block height out of range" error gracefully by checking if the node has synchronized to or beyond the required block height, otherwise without this validation the node would keep the network disabled if the user selected that option.
<details>
<summary>Provide a user-friendly message if the block height is out of range and exit the script cleanly.</summary>
```
/contrib/devtools/utxo_snapshot.sh 840000 snapshot2.dat ./src/bitcoin-cli -datadir=${AU_DATADIR}
Error: The node has not yet synchronized to block height 840001.
Please wait until the node has synchronized past this block height and try again.
```
</details>
ACKs for top commit:
achow101:
ACK 5b4f34006dbd76223b55b156421f87d2241ac296
fjahr:
tACK 5b4f34006dbd76223b55b156421f87d2241ac296
Tree-SHA512: 2b71286b627872d7cfdb367e29361afa3806a7ef9d65075b93892b735ff2ab729069e2f7259d30262909e73cef17fb7dca231615cc1863968cd042f4a2a4f901
37389c7d38 Merge bitcoin/bitcoin#28781: depends: latest config.guess & config.sub (fanquake)
3239f1525d Merge bitcoin/bitcoin#28479: build: use _LIBCPP_ENABLE_DEBUG_MODE over ENABLE_ASSERTIONS (fanquake)
45cc44bcf9 Merge bitcoin/bitcoin#27628: build: Fix shared lib linking for darwin with lld (fanquake)
b8ddcd937c Merge bitcoin/bitcoin#27575: Introduce platform-agnostic `ALWAYS_INLINE` macro (fanquake)
71c6d7f6ca Merge bitcoin/bitcoin#26653: test, init: perturb file to ensure failure instead of only deleting them (fanquake)
417f71a587 Merge bitcoin/bitcoin#27422: test: add coverage to rpc_scantxoutset.py (fanquake)
898dcbdc4f Merge bitcoin/bitcoin#27559: doc: clarify processing of mempool-msgs when NODE_BLOOM (glozow)
a4e429cb5a Merge bitcoin/bitcoin#26953: contrib: add ELF OS ABI check to symbol-check.py (fanquake)
deb7de26dd Merge bitcoin/bitcoin#26604: test: add coverage for `-bantime` (fanquake)
f725ed509a Merge bitcoin/bitcoin#26314: test: perturb anchors.dat to test error during initialization (fanquake)
3306f96d80 Merge bitcoin/bitcoin#25937: test: add coverage for rpc error when trying to rescan beyond pruned data (fanquake)
712dcaf86b Merge bitcoin/bitcoin#27516: test: simplify uint256 (de)serialization routines (fanquake)
90d65f25e1 Merge bitcoin/bitcoin#27508: build: use latest config.{guess,sub} in depends (fanquake)
df7be026e4 Merge bitcoin/bitcoin#27506: test: prevent intermittent failures (fanquake)
9b58b2d97b Merge bitcoin/bitcoin#27447: depends: Remove `_LIBCPP_DEBUG` from depends DEBUG mode (fanquake)
07770b77a1 Merge bitcoin/bitcoin#26741: doc: FreeBSD DataDirectoryGroupReadable Setting (fanquake)
5645362f11 Merge bitcoin/bitcoin#27362: test: remove `GetRNGState` lsan suppression (fanquake)
671e8e6851 Merge bitcoin/bitcoin#27368: refactor: Drop no longer used `CNetMsgMaker` instances (fanquake)
a11690bf62 Merge bitcoin/bitcoin#27301: depends: make fontconfig build under clang-16 (fanquake)
0a94b3f27b Merge bitcoin/bitcoin#27328: depends: fix osx build with clang 16 (fanquake)
Pull request description:
## Issue being fixed or feature implemented
Batch of trivial backports
## What was done?
See commits
## How Has This Been Tested?
built locally; large combined merge passed tests locally
## Breaking Changes
Should be none
## Checklist:
- [ ] I have performed a self-review of my own code
- [ ] I have commented my code, particularly in hard-to-understand areas
- [ ] I have added or updated relevant unit/integration/functional/e2e tests
- [ ] I have made corresponding changes to the documentation
- [x] I have assigned this pull request to a milestone _(for repository code-owners and collaborators only)_
ACKs for top commit:
UdjinM6:
utACK 37389c7d38
kwvg:
utACK 37389c7d38
Tree-SHA512: 1ea075a58361f57e037febcf003d380ab845b6c8e1c62d9fcf8954d46cd006046d1951f15a41a5deb9ab7af734df9dafcf89f33d115d78246752f7e2cd13f4ee
65ba8a79a2919a0bd89f2f2d981e072d4f2f549d contrib: add ELF ABI check to symbol-check.py (fanquake)
Pull request description:
Check that the operating system ABI version embedded into the release binaries, is the version we expect it to be.
ACKs for top commit:
laanwj:
Code review ACK 65ba8a79a2919a0bd89f2f2d981e072d4f2f549d
TheCharlatan:
ACK 65ba8a79a2919a0bd89f2f2d981e072d4f2f549d
Tree-SHA512: 798d7c3b05183becf113a2ea13d889e18f1cec01d3cc279e64dbddede4d57f87444978f3f52c44bc5fdf0ba93d77c7c0be37aa815f93f348c35da45dc3d30ac2
c75a0d4c57 Merge bitcoin/bitcoin#29177: build: Fix check whether `-latomic` needed (fanquake)
f670118cce Merge bitcoin/bitcoin#28851: build: Patch Qt to handle minimum macOS version properly (fanquake)
685ee8a46f Merge bitcoin/bitcoin#28884: doc: remove x86_64 build assumption from depends doc (fanquake)
47f6126504 Merge bitcoin/bitcoin#28881: doc: remove mention of missing bdb being a configure error (fanquake)
a9021db4ec Merge bitcoin/bitcoin#28777: doc: update docs for `CHECK_ATOMIC` macro (fanquake)
d5e15dfc5a Merge bitcoin/bitcoin#26839: Add support for RNDR/RNDRRS for AArch64 on Linux (Andrew Chow)
5aedcbfb43 Merge bitcoin/bitcoin#28778: depends: drop -O1 workaround from arm64 apple Qt build (fanquake)
95a8d8cfdc Merge bitcoin/bitcoin#21161: Fee estimation: extend bucket ranges consistently (glozow)
f4ea48e623 Merge bitcoin/bitcoin#28693: build: Include `config/bitcoin-config.h` explicitly in `util/trace.h` (fanquake)
f160e0dbb2 Merge bitcoin/bitcoin#28691: refactor: Remove CBlockFileInfo::SetNull (fanquake)
0278163aa3 Merge bitcoin/bitcoin#28697: fuzz: Increase merge -rss_limit_mb (fanquake)
90a1fb0e8d Merge bitcoin/bitcoin#28650: fuzz: Merge with -set_cover_merge=1 (fanquake)
f007abd19d Merge bitcoin/bitcoin#28459: build: add `-mbranch-protection=bti` (aarch64) to hardening flags (fanquake)
af8d12445a Merge bitcoin/bitcoin#28624: docs: fix typo (fanquake)
c740264da8 Merge bitcoin/bitcoin#28532: qt: enable` -ltcg` for windows under LTO (fanquake)
ccd3920d40 Merge bitcoin/bitcoin#28556: doc: fix link to developer-notes.md file in multiprocess.md (fanquake)
Pull request description:
## Issue being fixed or feature implemented
Batch of trivial backports
## What was done?
See commits
## How Has This Been Tested?
built locally; large combined merge passed tests locally
## Breaking Changes
Should be none
## Checklist:
_Go over all the following points, and put an `x` in all the boxes that apply._
- [ ] I have performed a self-review of my own code
- [ ] I have commented my code, particularly in hard-to-understand areas
- [ ] I have added or updated relevant unit/integration/functional/e2e tests
- [ ] I have made corresponding changes to the documentation
- [x] I have assigned this pull request to a milestone _(for repository code-owners and collaborators only)_
ACKs for top commit:
UdjinM6:
utACK c75a0d4c57
Tree-SHA512: 035dc3fa9812c7f381946ae4798b8e729a58b38a090d94502a8d992e9cfaab3307173c602d7b782c637a79c5c41b62570dc73bb4bb367e4505a039964926181b
8bf1d06599 Merge bitcoin/bitcoin#29308: doc: update `BroadcastTransaction` comment (glozow)
2a77808596 Merge bitcoin-core/gui#789: Avoid non-self-contained Windows header (Hennadii Stepanov)
da371b830d Merge bitcoin/bitcoin#28870: depends: Include `config.guess` and `config.sub` into `meta_depends` (fanquake)
2e41562d81 Merge bitcoin/bitcoin#29219: fuzz: Improve fuzzing stability for ellswift_roundtrip harness (fanquake)
b091329599 Merge bitcoin/bitcoin#29211: fuzz: fix `connman` initialization (Ava Chow)
df42d41060 Merge bitcoin/bitcoin#29200: net: create I2P sessions using both ECIES-X25519 and ElGamal encryption (fanquake)
4cdd1a8a5d Merge bitcoin/bitcoin#29172: fuzz: set `nMaxOutboundLimit` in connman target (fanquake)
97012ea522 Merge bitcoin/bitcoin#28962: doc: Rework guix docs after 1.4 release (fanquake)
c70ff5d702 Merge bitcoin/bitcoin#28844: contrib: drop GCC MAX_VERSION to 4.3.0 in symbol-check (fanquake)
e6f19e7760 Merge bitcoin/bitcoin#29068: test: Actually fail when a python unit test fails (fanquake)
75e0334866 Merge bitcoin/bitcoin#28989: test: Fix test by checking the actual exception instance (Andrew Chow)
8cd85d311f Merge bitcoin/bitcoin#28852: script, assumeutxo: Enhance validations in utxo_snapshot.sh (Ryan Ofsky)
fd2e88d6f3 Merge bitcoin/bitcoin#26077: guix: switch from `guix environment` to `guix shell` (fanquake)
02741a7706 Merge bitcoin/bitcoin#28913: coins: make sure PoolAllocator uses the correct alignment (fanquake)
dfd53dabed Merge bitcoin/bitcoin#28902: doc: Simplify guix install doc, after 1.4 release (fanquake)
Pull request description:
## Issue being fixed or feature implemented
Batch of trivial backports
## What was done?
See commits
## How Has This Been Tested?
built locally; large combined merge passed tests locally
## Breaking Changes
Should be none
## Checklist:
_Go over all the following points, and put an `x` in all the boxes that apply._
- [ ] I have performed a self-review of my own code
- [ ] I have commented my code, particularly in hard-to-understand areas
- [ ] I have added or updated relevant unit/integration/functional/e2e tests
- [ ] I have made corresponding changes to the documentation
- [x] I have assigned this pull request to a milestone _(for repository code-owners and collaborators only)_
ACKs for top commit:
UdjinM6:
utACK 8bf1d06599
knst:
utACK 8bf1d06599
Tree-SHA512: 506273e5a188f9ca74edf656e3cd338992192e6e97f68c89fc43e34be20fb7f211b48e4dfa8693727839a7920da8284509413c722f55774a428939c296dad517
8c3850923300352f14dc3dde6a6ce6689ddef185 contrib: move user32.dll from bitcoind.exe libs (fanquake)
Pull request description:
The user interface library is no-longer needed by `bitcoind.exe`, or utils, only `bitcoin-qt.exe`.
Add missing doc.
ACKs for top commit:
hebasto:
ACK 8c3850923300352f14dc3dde6a6ce6689ddef185, I've verified imported libraries on a Windows machine with the `dumpbin /imports` command.
Tree-SHA512: f752a5b807341c87320523f4e7c564c8acdbfc1313054a684844035102a7c4695d34cfefb0c6904f3151b2dfdcb54d6ea243c570deceeda30345944251e4c513
ff896d25819da1c1e80591595c922fb093942645 contrib: drop GCC MAX_VERSION to 4.3.0 in symbol-check (fanquake)
Pull request description:
Reflect the actual symbols used, i.e:
```bash
bitcoind: symbol __bswapsi2 from unsupported version GCC_4.3.0(7)
```
ACKs for top commit:
TheCharlatan:
ACK ff896d25819da1c1e80591595c922fb093942645
Tree-SHA512: b38ff8f4dd78d2d1c9063c53544dc4f240c3043f142e1581f7ba42f088a509293f6f17cc402c60ac82bff3b36668866b87e0e9e4d10d929484bb4c7a3e654f25
11b7269d83a56f919f9dddb7f7c72a96d428627f script: Enhance validations in utxo_snapshot.sh (pablomartin4btc)
Pull request description:
This PR resolves#27841 and some more:
- Ensure that the snapshot height is higher than the pruned block height when the node is pruned (Suggested by @Sjors [here](https://github.com/bitcoin/bitcoin/pull/28553#issuecomment-1804941396)).
- Validate the correctness of the file path and check if the file already exists (@hazeycode's [#27845](https://github.com/bitcoin/bitcoin/pull/27845)).
- Make network activity disablement optional for the user (Suggested by @Sjors [here](https://github.com/bitcoin/bitcoin/pull/16899#discussion_r342735815) and [here](https://github.com/bitcoin/bitcoin/pull/16899#issuecomment-536520911)).
- Ensure the `reconsiderblock` command is triggered on exit (@hazeycode's same PR as above), even in the case of user interruption (Ctrl-C).
In order to perform some testing please follow the instructions in the description of previous @hazeycode's PR #27845.
ACKs for top commit:
Sjors:
tACK 11b7269d83a56f919f9dddb7f7c72a96d428627f
ryanofsky:
Code review ACK 11b7269d83a56f919f9dddb7f7c72a96d428627f
Tree-SHA512: 2b699894c6f732ad5104f5a2bcf5dc86ed31edcc9d664690cab55b94a8ab00e2ca5bde901ee1d63acddca7ea80ad1734d8cfe78f9c02f8470f264fe93a2af759
This marks the completion of our transition from code based on
`talebook/statsd-client-cpp` to code based on `vthiery/cpp-statsd-client`.
Also, we long stopped using `snprintf`, remove it from exclusions list.
In upcoming commits, message sending will be split off into a separate
file and stats capabilities will be fleshed out. Prepare for that by
giving it its own directory.
Also get rid of `statsd` namespace, it is entirely unnecessary.
Co-authored-by: UdjinM6 <UdjinM6@users.noreply.github.com>
fa0074e2d82928016a43ca408717154a1c70a4db scripted-diff: Bump copyright headers (MarcoFalke)
Pull request description:
Needs to be done because no one has removed the years yet
ACKs for top commit:
practicalswift:
ACK fa0074e2d82928016a43ca408717154a1c70a4db
Tree-SHA512: 210e92acd7d400b556cf8259c3ec9967797420cfd19f0c2a4fa54cb2b3d32ad9ae27e771269201e7d554c0f4cd73a8b1c1a42c9f65d8685ca4d52e5134b071a3
1ef2138c0db3bd4f9332c777fa3fb2770dc1b08c lint: run mypy over contrib/devtools (fanquake)
Pull request description:
wumpus mentioned on IRC that we don't currently run `mypy` over the `contrib/devtools` directory, and that it would likely be worthwhile given #20434. This just adds that dir to the linter, as well as some missing annotations to fix existing errors. Note that now we require Python 3.6 we can make use of variable annotations.
master (patched to check contrib devtools):
```bash
test/lint/lint-python.sh
contrib/devtools/symbol-check.py:154: error: Incompatible types in assignment (expression has type "List[str]", variable has type "str")
contrib/devtools/circular-dependencies.py:35: error: Need type annotation for 'deps' (hint: "deps: Dict[<type>, <type>] = ...")
contrib/devtools/circular-dependencies.py:67: error: Need type annotation for 'closure' (hint: "closure: Dict[<type>, <type>] = ...")
Found 4 errors in 3 files (checked 187 source files)
```
I haven't quite gone as far as to add annotations like
```python
CHECKS: Dict[str, List[Tuple[str, Callable[[Any], bool]]]] = {...
```
to `symbol-check.py`.
ACKs for top commit:
laanwj:
ACK 1ef2138c0db3bd4f9332c777fa3fb2770dc1b08c
Tree-SHA512: a58c2ece588c640289dc1d35dad5b1b8732788272daa0965d6bf44ee8a7f7c8e8585f94d233ac41c84b9ffcfc97841a00fe2c9acba41f58fd164f01de4b6512b
this will allow us to rebase the PR before merging as we do now; and then right after merge it into develop and push
note we can not simply rebase locally before merging as we would then violate the "all changes must be done through PR rule"
When rebasing locally; we also check the range-diff (should be all >'s indicating a commit being pulled in from the rebase and ='s indicating the commits are the same as the base PR. This ensures that when we force push we will not invalidate previously created reviews.
Co-authored-by: UdjinM6 <UdjinM6@users.noreply.github.com>
80968cf scripted-diff: rename movie folder to animation (Peter Bushnell)
Pull request description:
Rename the movies directory and RES_MOVIES make variable to animation and RES_ANIMATION respectively. Movies is a bit of an unexpected term to be found.
ACKs for top commit:
MarcoFalke:
ACK 80968cf
hebasto:
ACK 80968cf, tested on Linux Mint 20 (Qt 5.12.8).
Tree-SHA512: 6bd31ce36e821f6a1bef8a7972086a2387d6258c48fc9df12d3ffdae07d0237036afbc2dec673384b78d9567b91d6e12eafa59fa2305aa79153dfd9b7c3a8655
ab9c34237ab7b056394e0bd1f7cb131ffd95754c release: remove gitian (fanquake)
Pull request description:
Note that this doesn't yet touch any glibc back compat related code.
ACKs for top commit:
laanwj:
Code review ACK ab9c34237ab7b056394e0bd1f7cb131ffd95754c
Tree-SHA512: 8e2fe3ec1097f54bb11ab9136b43818d90eab5dbb0a663ad6a552966ada4bdb49cc12ff4e66f0ec0ec5400bda5c81f3a3ce70a9ebb6fe1e0db612da9f00a51a7
fa4632c41714dfaa699bacc6a947d72668a4deef test: Move boost/stdlib includes last (MarcoFalke)
fa488f131fd4f5bab0d01376c5a5013306f1abcd scripted-diff: Bump copyright headers (MarcoFalke)
fac5c373006a9e4bcbb56843bb85f1aca4d87599 scripted-diff: Sort test includes (MarcoFalke)
Pull request description:
When writing tests, often includes need to be added or removed. Currently the list of includes is not sorted, so developers that write tests and have `clang-format` installed will either have an unrelated change (sorting) included in their commit or they will have to manually undo the sort.
This pull preempts both issues by just sorting all includes in one commit.
Please be aware that this is **NOT** a change to policy to enforce clang-format or any other developer guideline or process. Developers are free to use whatever tool they want, see also #18651.
Edit: Also includes a commit to bump the copyright headers, so that the touched files don't need to be touched again for that.
ACKs for top commit:
practicalswift:
ACK fa4632c41714dfaa699bacc6a947d72668a4deef
jonatack:
ACK fa4632c41714dfaa, light review and sanity checks with gcc build and clang fuzz build
Tree-SHA512: 130a8d073a379ba556b1e64104d37c46b671425c0aef0ed725fd60156a95e8dc83fb6f0b5330b2f8152cf5daaf3983b4aca5e75812598f2626c39fd12b88b180
6690adba08006739da0060eb4937126bdfa1181a Warn when binaries are built from a dirty branch. (Tyler Chambers)
Pull request description:
- Adjusted `--version` flag behavior in bitcoind and bitcoin-wallet to have the same behavior.
- Added `--version` flag to bitcoin-tx to match.
- Added functionality in gen-manpages.sh to error when attempting to generate man pages for binaries built from a dirty branch.
mitigates problem with issue #20412
ACKs for top commit:
laanwj:
Tested ACK 6690adba08006739da0060eb4937126bdfa1181a
Tree-SHA512: b5ca509f1a57f66808c2bebc4b710ca00c6fec7b5ebd7eef58018e28e716f5f2358e36551b8a4df571bf3204baed565a297aeefb93990e7a99add502b97ee1b8
note: focal uses glibc 2.31, which is used by gitian, so our build
environment has support for a version of glibc that far exceeds
our maximum supported version (2.28). this could be remedied by
switching to debian buster (oldstable) which uses 2.28.
e09c701e0110350f78366fb837308c086b6503c0 scripted-diff: Bump copyright of files changed in 2020 (MarcoFalke)
6cbe6209646db8914b87bf6edbc18c6031a16f1e scripted-diff: Replace CCriticalSection with RecursiveMutex (MarcoFalke)
Pull request description:
`RecursiveMutex` better clarifies that the mutex is recursive, see also the standard library naming: https://en.cppreference.com/w/cpp/thread/recursive_mutex
For that reason, and to avoid different people asking me the same question repeatedly (e.g. https://github.com/bitcoin/bitcoin/pull/15932#pullrequestreview-339175124 ), remove the outdated alias `CCriticalSection` with a scripted-diff
