Commit Graph

24184 Commits

Author SHA1 Message Date
Konstantin Akimov
fd2e9857aa fix: follow-up partial bitcoin/bitcoin#25063 - actually load binaries with x86_64-apple-darwin platform 2023-11-24 11:23:46 -06:00
fanquake
007076d372 Merge bitcoin/bitcoin#24603: macdeploy: remove unused detached-sig-apply.sh
979271a5d9ff887cb2efb199feaf9602c9b2086d macdeploy: remove unused detached-sig-apply (fanquake)

Pull request description:

  Signature application is now done with signapple.

  8435d7f11a/contrib/guix/libexec/codesign.sh (L84-L85)

ACKs for top commit:
  laanwj:
    ACK 979271a5d9ff887cb2efb199feaf9602c9b2086d
  gruve-p:
    ACK 979271a5d9
  achow101:
    ACK 979271a5d9ff887cb2efb199feaf9602c9b2086d
  hebasto:
    ACK 979271a5d9ff887cb2efb199feaf9602c9b2086d, I have reviewed the code and it looks OK, I agree it can be merged.

Tree-SHA512: ab51a609d00cead4f33bcfc5b5ff1008ee02363ab1f4c4bf9544631069c237bfa92eac4dfa231bff8a1d702bda6cc92b4151361f74f58e77b595e0cb82a8391a
2023-11-24 11:23:46 -06:00
MarcoFalke
d7bdf8e424 Merge bitcoin/bitcoin#26694: test: get_previous_releases.py: M1/M2 macs can't run unsigned arm64 binaries; self-sign when needed
dc12f2e212dfacbe238cf68eb454b9ec71169bbc test: improve error msg on previous release tarball extraction failure (kdmukai)
7121fd8fa7de50ff67157f81f9e0f267b9795dbb test: self-sign previous release binaries for arm64 macOS (kdmukai)

Pull request description:

  ## The Problem
  If you run `test/get_previous_releases.py -b` on an M1 or M2 mac, you'll get an unsigned v23.0 binary in the arm64 tarball. macOS [sets stricter requirements on ARM binaries](https://news.ycombinator.com/item?id=26996578) so the unsigned arm64 binary is apparently completely unusable without being signed/notarized(?).

  This means that any test that depends on a previous release (e.g. `wallet_backwards_compatibility.py`) will fail because the v23.0 node cannot launch:

  ```
  TestFramework (ERROR): Assertion failed
  Traceback (most recent call last):
    File "/Users/kdmukai/dev/bitcoin-core/test/functional/test_framework/test_framework.py", line 563, in start_nodes
      node.wait_for_rpc_connection()
    File "/Users/kdmukai/dev/bitcoin-core/test/functional/test_framework/test_node.py", line 231, in wait_for_rpc_connection
      raise FailedToStartError(self._node_msg(
  test_framework.test_node.FailedToStartError: [node 2] bitcoind exited with status -9 during initialization
  ```

  This can also be confirmed by downloading bitcoin-23.0-arm64-apple-darwin.tar.gz (https://bitcoincore.org/bin/bitcoin-core-23.0/) and trying to run any of the binaries manually on an M1 or M2 mac.

  ## Solution in this PR
  (UPDATED) Per @ hebasto, we can self-sign the arm64 binaries. This PR checks each binary in the previous release's "bin/" and verifies if the arm64 binary is signed. If not, attempt to self-sign and confirm success.

  (note: an earlier version of this PR downloaded the x86_64 binary as a workaround but this approach has been discarded)

  ## Longer term solution
  If possible, produce signed arm64 binaries in a future v23.x tarball?

  Note that this same problem affects the new v24.0.1 arm64 tarball so perhaps a signed v24.x.x tarball would also be ideal?

  That being said, this PR will check all current and future arm64 binaries and self-sign as needed, so perhaps we need not worry about pre-signing the tarball binaries. And I did test a version of `get_previous_releases.py` that includes the new v24.0.1 binaries and it successfully self-signed both v23.0 and v24.0.1, as expected.

  ## Further info:
  Somewhat related to: https://github.com/bitcoin/bitcoin/issues/15774#issuecomment-1265164753

  And @ fanquake noted on IRC that you can confirm which binaries are or are not signed via:
  ```
  $ codesign -v -d bitcoin-qt
  bitcoin-qt: code object is not signed at all
  ```

ACKs for top commit:
  hebasto:
    ACK dc12f2e212dfacbe238cf68eb454b9ec71169bbc

Tree-SHA512: 644895f8e97f5ffb3c4754c1db2c48abd77fa100c2058e3c896af04806596fc2b9c807a3f3a2add5be53301ad40ca2b8171585bd254e691f6eb38714d938396b
2023-11-24 11:23:46 -06:00
MacroFake
d5654db27e partial Merge bitcoin/bitcoin#25063: test: previous releases: add v23.0
dba123167236a172d2d33861d58aa94a19729671 test: previous releases: add v23.0 (Sjors Provoost)

Pull request description:

  Follows the same pattern as d8b705f1caeb3b4a6790cb26e4e5584ca791d965 (v22.0) and 8a57a06a5062dd8dfdefca4e404d0ddbd2a3da1d (v0.21.0).

  Starting from v23.0 there is a separate macOS release for x86_64 and aarch64.

ACKs for top commit:
  prusnak:
    Approach ACK dba123167236a172d2d33861d58aa94a19729671

Tree-SHA512: 249aeddd5e80e163578581e5c8e9b6579f3694abc3d1fb68dddb7b42d75021ad85266688ec4a365a6631d82a65a19873aff7ba61c0ea59d21f8adbe4b772dc16
2023-11-24 11:23:46 -06:00
laanwj
8b61966944 Merge bitcoin/bitcoin#24669: build: further consolidate macOS deployment
3d415215699e718b3f6eea6e3c9fb2948476f930 build: perform /Applications symlink generation in macdeployqtplus (fanquake)
dac693671928aa3fc304e6a802abfffb2f4ec8fd build: perform all .tiff copying in macdeployqtplus (fanquake)

Pull request description:

  Rather than maintaining 2 different versions of the same code (`.tiff` copying and symlink generation), consolidate to just the Python code, and use it on macOS and Linux. Previously Linux would  perform the 2 actions in the makefile, and then would still be running the `macdeployqtplus` script, so it makes sense to further consolidate deployment operations into the script.

  Guix Build (on x86_64):
  ```bash
  23343f04c426c7ff078afae4e600a7028970d4d86eed8b7834696d9e4d684151  guix-build-3d415215699e/output/arm64-apple-darwin/SHA256SUMS.part
  c28b2a2e4888bf84369aa25804e2576347d5ab09416354ec8b95c76a9d38ff96  guix-build-3d415215699e/output/arm64-apple-darwin/bitcoin-3d415215699e-arm64-apple-darwin-unsigned.dmg
  9a57077b2bd722a7d85d26b66cbce5abdb791985fe9d9d37e884c79ba8751e24  guix-build-3d415215699e/output/arm64-apple-darwin/bitcoin-3d415215699e-arm64-apple-darwin-unsigned.tar.gz
  d2b06dc5b86541798ace41dab569849f7403e7ff9ec329bda671ec84e6fad549  guix-build-3d415215699e/output/arm64-apple-darwin/bitcoin-3d415215699e-arm64-apple-darwin.tar.gz
  608e7d51a44ab9c5b28eb3703a0f4fe98b4adff22c77a5502786b84bd96cc188  guix-build-3d415215699e/output/dist-archive/bitcoin-3d415215699e.tar.gz
  3e483705b1f9f1fb8f6afedc8ad0214a6cb00e77f766c0b03c42d56f410d4362  guix-build-3d415215699e/output/x86_64-apple-darwin/SHA256SUMS.part
  9370e3e3b7d47b5a44e64554cf3b6d7e0671b072c08cd251eacc7ec72ce2b53f  guix-build-3d415215699e/output/x86_64-apple-darwin/bitcoin-3d415215699e-x86_64-apple-darwin-unsigned.dmg
  ad0f68682d78c311497669fc3d627138be37510215d259b5f0b686d93e7d83b7  guix-build-3d415215699e/output/x86_64-apple-darwin/bitcoin-3d415215699e-x86_64-apple-darwin-unsigned.tar.gz
  e09dce4ff692ef66d1f4818083c1880bcf3a79c53112561d9e929bb6e5ffc011  guix-build-3d415215699e/output/x86_64-apple-darwin/bitcoin-3d415215699e-x86_64-apple-darwin.tar.gz
  ```

ACKs for top commit:
  laanwj:
    Re-ACK 3d415215699e718b3f6eea6e3c9fb2948476f930

Tree-SHA512: 80dd66a6e94c5b3e8823ccb57dcb08a8851a1e70a154b62385443f8d2d5ed5af900a0ac5003143959863586f1c7b90002fe6bff3ca5e37697253e051f69d7629
2023-11-24 11:23:46 -06:00
fanquake
23adfb7bfd Merge bitcoin/bitcoin#22546: build, qt: Fix make deploy on M1-based macOS with system frameworks
1513727e2b38800c694d1204cb454cc6fabc4937 build, qt: (Re-)sign package (Hennadii Stepanov)
c26a0a5af76bed9c2eb65f1a19725508c55299e8 build, qt: Align frameworks with macOS codesign tool requirements (Hennadii Stepanov)

Pull request description:

  Fixes #22403

  This PR follows Apple [docs](https://developer.apple.com/documentation/macos-release-notes/macos-big-sur-11_0_1-universal-apps-release-notes):
  >  - New in macOS 11 on Macs with Apple silicon, and starting in macOS Big Sur 11 beta 6, the operating system enforces that any executable must be signed before it’s allowed to run. There isn’t a specific identity requirement for this signature: a simple ad-hoc signature is sufficient...
  >  - ... If you use a custom workflow involving tools that modify a binary after linking (e.g. `strip` or `install_name_tool`) you might need to manually call `codesign` as an additional build phase to properly ad-hoc sign your binary. These new signatures are not bound to the specific machine that was used to build the executable, they can be verified on any other system and will be sufficient to comply with the new default code signing requirement on Macs with Apple silicon...

  When building with system Qt frameworks (i.e., without depends), a new string has been added to the `make deploy` log on M1-based macOS:
  ```
  % make deploy
  ...
  + Generating .DS_Store +
  dist/Bitcoin-Qt.app: replacing existing signature
  + Preparing .dmg disk image +
  ...
  ```

  This PR does not change build system behavior:
  - when building with depends
  - on Intel-based macOS

ACKs for top commit:
  jarolrod:
    ACK 1513727e2b38800c694d1204cb454cc6fabc4937
  fanquake:
    ACK 1513727e2b38800c694d1204cb454cc6fabc4937 - although didn't test on M1 hardware. Given the forced signing is scoped to only occur when running the deploy script on macOS, this doesn't interfere with our release signing.

Tree-SHA512: 3aa778fdd6ddb54f029f632f2fe52c2ae3bb197ba564cb776493aa5c3a655bd51d10ccbe6c007372d717e9b01fc4193dd5c29ea0bc7e069dcae7e991ae259f0c
2023-11-24 11:23:46 -06:00
fanquake
736b7a47f1 Merge bitcoin/bitcoin#22199: macdeploy: minor fixups and simplifications
0a5723beea9c909b437e8c3fa434506019c1198c macdeploy: cleanup .temp.dmg if present (fanquake)
ecffe8689dfbdc33deba8119376dcc8f208f0f72 macdeploy: remove qt4 related code (fanquake)
639f0642539c6b5ba9bc7b39bb8bb52752029bee macdeploy: select the plugins we need, rather than excluding those we don't (fanquake)
3d26b6b9e928e3cdc4b3d8d1f66ec7ed022b411b macdeploy: fix framework printing when passing -verbose (fanquake)
dca6c9032993f2bbf8047751d52f2a5c7ebd3ee4 macdeploy: remove unused plistlib import (fanquake)

Pull request description:

  This includes [one followup](https://github.com/bitcoin/bitcoin/pull/20422#discussion_r534207899) and [one bug fix](3d26b6b9e9) from #20422, as well as some simplifications to the `macdeployqtplus` code.

ACKs for top commit:
  hebasto:
    ACK 0a5723beea9c909b437e8c3fa434506019c1198c, tested on macOS Big Sur 11.4 (20F71, x86_64) + Homebrew's Qt 5.15.2.

Tree-SHA512: cfad9505eacd32fe3a9d06eb13b2de0b6d2cad7b17778e90b503501cbf922e53d4e7f7f74952d1aed58410bdae9b0bb3248098583ef5b85689cb27d4dc06c029
2023-11-24 11:23:46 -06:00
fanquake
ed8da9277a Merge bitcoin/bitcoin#25817: build: fix cross-arch macOS M1 build reproducibility
08bd3382777bf5d24fa8f32f27abc7b6d1e20dd9 build: optimise arm64 darwin qt build using -O1 (fanquake)

Pull request description:

  Building the macOS M1 bitcoin-qt binary at a optimisation level higher than `-O1` causes reproducibility issues when building on different architectures.

  Proposing somewhat of a hammer.

  This would fix 1 of the 2 remaining HOSTS in #21194.

  Guix Build (x86_64):
  ```bash
  1b58b5109b32dca2509499c93347148e6bab5dca835081f8cbd3123bed72cce1  guix-build-08bd3382777b/output/arm64-apple-darwin/SHA256SUMS.part
  0e0d063d3832fad7c5116dabb2ac33c919f40bda04759aad4523c6247295bc9e  guix-build-08bd3382777b/output/arm64-apple-darwin/bitcoin-08bd3382777b-arm64-apple-darwin-unsigned.dmg
  1feb301245f2c664edcfd9ac528fe1543fc7b183b3b42637db77d57658bc2b5e  guix-build-08bd3382777b/output/arm64-apple-darwin/bitcoin-08bd3382777b-arm64-apple-darwin-unsigned.tar.gz
  641eb100d0a281203f9d6e36e45dc0ffc772c680d6aec462434f106b4c44e295  guix-build-08bd3382777b/output/arm64-apple-darwin/bitcoin-08bd3382777b-arm64-apple-darwin.tar.gz
  9d89920626e35939aa6cf506fc85861179f3c0e18d4ef1954750cf81336a851a  guix-build-08bd3382777b/output/dist-archive/bitcoin-08bd3382777b.tar.gz
  e7697d30084270d0b5843b3baf0d752e240c2f708f728bc2f6896f153276ca6b  guix-build-08bd3382777b/output/x86_64-apple-darwin/SHA256SUMS.part
  dd77acee082dbfd3cdad2c564bbd3bdace8df9bf32f92cf4a2debd5a996ace49  guix-build-08bd3382777b/output/x86_64-apple-darwin/bitcoin-08bd3382777b-x86_64-apple-darwin-unsigned.dmg
  85e63fccb7af12468a04a678034c42dcd775d243b2d194a52e1086a6ffbdbe84  guix-build-08bd3382777b/output/x86_64-apple-darwin/bitcoin-08bd3382777b-x86_64-apple-darwin-unsigned.tar.gz
  ff2629957608898d76a42025985e3ec4bf5dc8572794e32b4182ba6f8babb828  guix-build-08bd3382777b/output/x86_64-apple-darwin/bitcoin-08bd3382777b-x86_64-apple-darwin.tar.gz
  ```

  Guix Build (arm64):
  ```bash
  1b58b5109b32dca2509499c93347148e6bab5dca835081f8cbd3123bed72cce1  guix-build-08bd3382777b/output/arm64-apple-darwin/SHA256SUMS.part
  0e0d063d3832fad7c5116dabb2ac33c919f40bda04759aad4523c6247295bc9e  guix-build-08bd3382777b/output/arm64-apple-darwin/bitcoin-08bd3382777b-arm64-apple-darwin-unsigned.dmg
  1feb301245f2c664edcfd9ac528fe1543fc7b183b3b42637db77d57658bc2b5e  guix-build-08bd3382777b/output/arm64-apple-darwin/bitcoin-08bd3382777b-arm64-apple-darwin-unsigned.tar.gz
  641eb100d0a281203f9d6e36e45dc0ffc772c680d6aec462434f106b4c44e295  guix-build-08bd3382777b/output/arm64-apple-darwin/bitcoin-08bd3382777b-arm64-apple-darwin.tar.gz
  9d89920626e35939aa6cf506fc85861179f3c0e18d4ef1954750cf81336a851a  guix-build-08bd3382777b/output/dist-archive/bitcoin-08bd3382777b.tar.gz
  e7697d30084270d0b5843b3baf0d752e240c2f708f728bc2f6896f153276ca6b  guix-build-08bd3382777b/output/x86_64-apple-darwin/SHA256SUMS.part
  dd77acee082dbfd3cdad2c564bbd3bdace8df9bf32f92cf4a2debd5a996ace49  guix-build-08bd3382777b/output/x86_64-apple-darwin/bitcoin-08bd3382777b-x86_64-apple-darwin-unsigned.dmg
  85e63fccb7af12468a04a678034c42dcd775d243b2d194a52e1086a6ffbdbe84  guix-build-08bd3382777b/output/x86_64-apple-darwin/bitcoin-08bd3382777b-x86_64-apple-darwin-unsigned.tar.gz
  ff2629957608898d76a42025985e3ec4bf5dc8572794e32b4182ba6f8babb828  guix-build-08bd3382777b/output/x86_64-apple-darwin/bitcoin-08bd3382777b-x86_64-apple-darwin.tar.gz
  ```

ACKs for top commit:
  hebasto:
    ACK 08bd3382777bf5d24fa8f32f27abc7b6d1e20dd9
  jarolrod:
    ACK 08bd3382777bf5d24fa8f32f27abc7b6d1e20dd9

Tree-SHA512: 48da4acb1799c3153cdaf674f287c81c3da230a3476183616b74f318baa595af45b313136eb228ba13c63e0b8206a78064734f9fd0488e1e839c9e4e1d92ba25
2023-11-24 11:23:46 -06:00
UdjinM6
1d9991fe24
fix: pass GITHUB_REPOSITORY into Dockerfile.GitHubActions.Release (#5724)
## Issue being fixed or feature implemented
Should hopefully fix
https://github.com/dashpay/dash-dev-branches/actions/runs/6939402277/job/18876687119

#5716 follow-up

## What was done?
`$GITHUB_REPOSITORY` is not available inside docker, pass it inside

## How Has This Been Tested?

## Breaking Changes
n/a

## Checklist:
- [x] I have performed a self-review of my own code
- [ ] I have commented my code, particularly in hard-to-understand areas
- [ ] I have added or updated relevant unit/integration/functional/e2e
tests
- [ ] I have made corresponding changes to the documentation
- [x] I have assigned this pull request to a milestone _(for repository
code-owners and collaborators only)_
2023-11-24 11:23:10 -06:00
PastaPastaPasta
6db3c13bac
Merge pull request #5721 from UdjinM6/merge_master_20.0.1
chore: Merge master 20.0.1 back into develop
2023-11-21 16:52:24 -06:00
PastaPastaPasta
39ce20cddc
Merge pull request #5575 from kwvg/secp256k1
backport: merge bitcoin#27479, #27230, #25251, partial #22934, #23383, #24792, #26691, #27445 (secp256k1 update)
2023-11-21 16:48:00 -06:00
Kittywhiskers Van Gogh
00b3dd6690 ci: don't re-use config.cache when running configure script 2023-11-21 07:59:03 -06:00
Kittywhiskers Van Gogh
78a9f1e55b merge bitcoin#27479: BIP324: ElligatorSwift integrations 2023-11-21 07:59:03 -06:00
Kittywhiskers Van Gogh
ccc9be4966 trivial: add header for std::array instantiation 2023-11-21 07:59:03 -06:00
Kittywhiskers Van Gogh
4ac25312cc partial bitcoin#27445: Update src/secp256k1 subtree to release v0.3.1
excludes:
- 719a74989be3cfbc4422ec07cac199c295d28d05
- 621c17869d3754559c03e4f2bee73885659e0c68
2023-11-21 07:59:03 -06:00
Kittywhiskers Van Gogh
2458280f41 merge bitcoin#27230: Update src/secp256k1 subtree to upstream release v0.3.0 2023-11-21 07:59:03 -06:00
Kittywhiskers Van Gogh
a3f29982ad partial bitcoin#26691: Update secp256k1 subtree to libsecp256k1 version 0.2.0
notes:
- excludes changes made to `SignSchnorr`, `XOnlyPubKey`, kernel context and absent {fuzz,bench} tests
2023-11-21 07:59:03 -06:00
Kittywhiskers Van Gogh
e2f576b3c8 merge bitcoin#25251: Consolidate Windows ASLR workarounds for upstream secp256k1 changes 2023-11-21 07:59:03 -06:00
Kittywhiskers Van Gogh
ac140f0299 partial bitcoin#24792: Update libsecp256k1 subtree to current master
excludes:
- d960d4fd3a767cf5695bed96c5f329056f77d0da
- 404c53062bb80853d5967187bdb7b5f7e749de7f
2023-11-21 07:59:03 -06:00
Kittywhiskers Van Gogh
7e6213b0ff partial bitcoin#23383: Update libsecp256k1 subtree to current master
excludes
- 314195c8be3bd7db0d5817c4fb3aa85c84363ce9
2023-11-21 07:59:03 -06:00
Kittywhiskers Van Gogh
fecccbf94d build: add --enable-module-schnorrsig --enable-experimental cfg flags
taken from changes made to `configure.ac` in 0664f5fe1f77f08d235aa3750b59428257b0b91d (bitcoin#19953)
2023-11-21 07:59:03 -06:00
Kittywhiskers Van Gogh
a3e6aeddde partial bitcoin#22934: Add verification to Sign, SignCompact and SignSchnorr
notes:
- excludes changes done to `SignSchnorr`
2023-11-21 07:59:03 -06:00
UdjinM6
9861bc74d6
fix: should avoid implicit conversions in pushKV params (#5719)
## Issue being fixed or feature implemented
Should fix compilation errors like
```
masternode/meta.cpp:43:9: error: call to member function 'pushKV' is ambiguous
    ret.pushKV("lastOutboundAttemptElapsed", now - lastOutboundAttempt);
    ^~
masternode/meta.cpp:45:9: error: call to member function 'pushKV' is ambiguous
    ret.pushKV("lastOutboundSuccessElapsed", now - lastOutboundSuccess);
    ^~
```
on FreeBSD + clang-15

kudos to @MrDefacto for finding the issue and testing the fix


## What was done?
Specify `now` variable type explicitly instead of relying on `auto`


## How Has This Been Tested?
MrDefacto confirmed it compiles with no issues on FreeBSD now

## Breaking Changes
n/a

## Checklist:
- [x] I have performed a self-review of my own code
- [ ] I have commented my code, particularly in hard-to-understand areas
- [ ] I have added or updated relevant unit/integration/functional/e2e
tests
- [ ] I have made corresponding changes to the documentation
- [x] I have assigned this pull request to a milestone _(for repository
code-owners and collaborators only)_
2023-11-21 07:53:55 -06:00
PastaPastaPasta
1908f51baa
ci: change Dockerfile.GitHubActions.Release to use local repo and not always dashpay/dash (#5716)
## Issue being fixed or feature implemented
In order to provide nightly builds over at dash-dev-branches we need to
be able to run this automation with other REPOs

## What was done?
Make it repo specific.

## How Has This Been Tested?
Hasn't yet

## Breaking Changes
None

## Checklist:
_Go over all the following points, and put an `x` in all the boxes that
apply._
- [x] I have performed a self-review of my own code
- [ ] I have commented my code, particularly in hard-to-understand areas
- [ ] I have added or updated relevant unit/integration/functional/e2e
tests
- [ ] I have made corresponding changes to the documentation
- [x] I have assigned this pull request to a milestone _(for repository
code-owners and collaborators only)_
2023-11-20 22:15:07 -06:00
UdjinM6
64e62560cd
Merge branch 'master' into merge_master_20.0.1 2023-11-20 20:07:12 +03:00
Odysseas Gabrielides
112564974d
refactor: deprecate non-deterministic IS support (#5553)
## Issue being fixed or feature implemented
Non-deterministic IS locks aren't used anymore since v18 dip24.
We should drop that support to make code simpler.

## What was done?
Dropped non-deterministic IS code, `evo_instantsend_tests` and
`feature_llmq_is_migration.py` (don't need it anymore), adjusted func
tests.

## How Has This Been Tested?
all tests, synced Testnet

## Breaking Changes

## Checklist:
- [x] I have performed a self-review of my own code
- [x] I have commented my code, particularly in hard-to-understand areas
- [ ] I have added or updated relevant unit/integration/functional/e2e
tests
- [ ] I have made corresponding changes to the documentation
- [x] I have assigned this pull request to a milestone _(for repository
code-owners and collaborators only)_

---------

Co-authored-by: UdjinM6 <UdjinM6@users.noreply.github.com>
Co-authored-by: Konstantin Akimov <545784+knst@users.noreply.github.com>
2023-11-20 10:17:04 -06:00
UdjinM6
531dcad955
fix: avoid crashes on "corrupted db" reindex attempts (#5717)
## Issue being fixed or feature implemented
Should fix crashes like
```
: Corrupted block database detected.
Please restart with -reindex or -reindex-chainstate to recover.
Assertion failure:
  assertion: globalInstance == nullptr
  file: mnhftx.cpp, line: 43
  function: CMNHFManager
   0#: (0x105ADA27C) stacktraces.cpp:629  - __assert_rtn
   1#: (0x104945794) mnhftx.cpp:43        - CMNHFManager::CMNHFManager(CEvoDB&)
   2#: (0x10499DA90) compressed_pair.h:40 - std::__1::__unique_if<CMNHFManager>::__unique_single std::__1::make_unique[abi:v15006]<CMNHFManager, CEvoDB&>(CEvoDB&)
   3#: (0x10499753C) init.cpp:1915        - AppInitMain(std::__1::variant<std::__1::nullopt_t, std::__1::reference_wrapper<NodeContext>, std::__1::reference_wrapper<WalletContext>, std::__1::reference_wrapper<CTxMemPool>, std::__1::reference_wrapper<ChainstateManager>, std::__1::reference_wrapper<CBlockPolicyEstimator>, std::__1::reference_wrapper<LLMQContext>> const&, NodeContext&, interfaces::BlockAndHeaderTipInfo*)
```

## What was done?


## How Has This Been Tested?

## Breaking Changes
n/a

## Checklist:
- [x] I have performed a self-review of my own code
- [ ] I have commented my code, particularly in hard-to-understand areas
- [ ] I have added or updated relevant unit/integration/functional/e2e
tests
- [ ] I have made corresponding changes to the documentation
- [x] I have assigned this pull request to a milestone _(for repository
code-owners and collaborators only)_
2023-11-20 10:08:48 -06:00
PastaPastaPasta
722d14c340
Merge pull request #5702 from ogabrielides/bitcoin_backport_24946
backport: Merge bitcoin#24946, #22649, #22704, #23806, #26153
2023-11-19 10:20:32 -06:00
fanquake
5a4406ef98 Merge bitcoin/bitcoin#26153: Reduce wasted pseudorandom bytes in ChaCha20 + various improvements
511aa4f1c7508f15cab8d7e58007900ad6fd3d5d Add unit test for ChaCha20's new caching (Pieter Wuille)
fb243d25f754da8f01793b41e2d225b917f3e5d7 Improve test vectors for ChaCha20 (Pieter Wuille)
93aee8bbdad808b7009279b67470d496cc26b936 Inline ChaCha20 32-byte specific constants (Pieter Wuille)
62ec713961ade7b58e90c905395558a41e8a59f0 Only support 32-byte keys in ChaCha20{,Aligned} (Pieter Wuille)
f21994a02e1cc46d41995581b54222abc655be93 Use ChaCha20Aligned in MuHash3072 code (Pieter Wuille)
5d16f757639e2cc6e81db6e07bc1d5dd74abca6c Use ChaCha20 caching in FastRandomContext (Pieter Wuille)
38eaece67b1bc37b2f502348c5d7537480a34346 Add fuzz test for testing that ChaCha20 works as a stream (Pieter Wuille)
5f05b27841af0bed1b6e7de5f46ffe33e5919e4d Add xoroshiro128++ PRNG (Martin Leitner-Ankerl)
12ff72476ac0dbf8add736ad3fb5fad2eeab156c Make unrestricted ChaCha20 cipher not waste keystream bytes (Pieter Wuille)
6babf402130a8f3ef3058594750aeaa50b8f5044 Rename ChaCha20::Seek -> Seek64 to clarify multiple of 64 (Pieter Wuille)
e37bcaa0a6dbb334ab6e817efcb609ccee6edc39 Split ChaCha20 into aligned/unaligned variants (Pieter Wuille)

Pull request description:

  This is an alternative to #25354 (by my benchmarking, somewhat faster), subsumes #25712, and adds additional test vectors.

  It separates the multiple-of-64-bytes-only "core" logic (which becomes simpler) from a layer around which performs caching/slicing to support arbitrary byte amounts. Both have their uses (in particular, the MuHash3072 code can benefit from multiple-of-64-bytes assumptions), plus the separation results in more readable code. Also, since FastRandomContext effectively had its own (more naive) caching on top of ChaCha20, that can be dropped in favor of ChaCha20's new built-in caching.

  I thought about rebasing #25712 on top of this, but the changes before are fairly extensive, so redid it instead.

ACKs for top commit:
  ajtowns:
    ut reACK 511aa4f1c7508f15cab8d7e58007900ad6fd3d5d
  dhruv:
    tACK crACK 511aa4f1c7

Tree-SHA512: 3aa80971322a93e780c75a8d35bd39da3a9ea570fbae4491eaf0c45242f5f670a24a592c50ad870d5fd09b9f88ec06e274e8aa3cefd9561d623c63f7198cf2c7
2023-11-19 10:20:12 -06:00
MarcoFalke
1915914e5b Merge bitcoin/bitcoin#23806: fuzz: follow up for #22704
8f79831ab57b8fce48bb7b01fce86fac338755a5 Refactor the chacha20 differential fuzz test (stratospher)

Pull request description:

  This PR addresses [comments from #22704](https://github.com/bitcoin/bitcoin/pull/22704/files#discussion_r771510963)  to make the following changes in `src/test/fuzz/crypto_diff_fuzz_chacha20.cpp`:

  - replace `memcmp()` with ==
  - add a missing assert statement to compare the encrypted bytes

Top commit has no ACKs.

Tree-SHA512: 02338460fb3a89e732558bf00f3aebf8f04daba194e03ae0e3339bb2ff6ba35d06841452585b739047a29f8ec64f36b1b4ce2dfa39a08f6ad44a6a937e7b3acb
2023-11-19 10:20:12 -06:00
W. J. van der Laan
c8650ec003 Merge bitcoin/bitcoin#22704: fuzz: Differential fuzzing to compare Bitcoin Core's and D. J. Bernstein's implementation of ChaCha20
4d0ac72f3ae78e3c6a0d5dc4f7e809583abd0546 [fuzz] Add fuzzing harness to compare both implementations of ChaCha20 (stratospher)
65ef93203cc6a977c8e96f07cb9155f46faf5004 [fuzz] Add D. J. Bernstein's implementation of ChaCha20 (stratospher)

Pull request description:

  This PR compares Bitcoin Core's implementation of ChaCha20 with D. J. Bernstein's in order to find implementation discrepancies if any.

ACKs for top commit:
  laanwj:
    Code review ACK 4d0ac72f3ae78e3c6a0d5dc4f7e809583abd0546

Tree-SHA512: f826144b4db61b9cbdd7efaaca8fa9cbb899953065bc8a26820a566303b2ab6a17431e7c114635789f0a63fbe3b65cb0bf2ab85baf882803a5ee172af4881544
2023-11-19 10:20:12 -06:00
MarcoFalke
47828bd76b Merge bitcoin/bitcoin#22649: fuzz: Avoid OOM in system fuzz target
fa7718344d2879bb3f3c00a4185c5445390c017d fuzz: Avoid OOM in system fuzz target (MarcoFalke)

Pull request description:

  If the inputs size is unlimited, the target may consume unlimited memory, because the argsmanager stores the argument names. Limiting the size should fix this issue.

  Should fix https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=36906

ACKs for top commit:
  practicalswift:
    cr ACK fa7718344d2879bb3f3c00a4185c5445390c017d

Tree-SHA512: 6edfcf324ee9d94e511038ee01340f02db50bcb233af3f1a1717c3602164c88528d9d987e971ec32f1a4593b868019bea0102c53c9b02bfefec3dfde959483cf
2023-11-19 10:20:12 -06:00
MacroFake
e98e5dc740 Merge bitcoin/bitcoin#24946: Unroll the ChaCha20 inner loop for performance
81c09ee45caecf8d9daf6766b94cebf54f3f08cd Unroll the ChaCha20 inner loop for performance (Pieter Wuille)

Pull request description:

  Unrolling the inner ChaCha20 loop gives a ~15% speedup for me in the CHACHA20_* benchmarks. It's a simple change, this performance helps with RNG generation, and will matter more for BIP324.

ACKs for top commit:
  martinus:
    tested ACK  81c09ee with clang++ 13.0.1, test `CHACHA20_1MB`:
  MarcoFalke:
    ACK 81c09ee45caecf8d9daf6766b94cebf54f3f08cd 🍟

Tree-SHA512: 108bd0ba573bb08de92d611e7be7c09a2c2700f9655f44129b87f9b71f7e101dfc6bd345783e7b4b9b40f0b003913cf59187f422da8cdb5b20887f7855b2611a
2023-11-19 10:20:12 -06:00
PastaPastaPasta
105acfd5fc
Merge pull request #5698 from knst/v20-fix-todo-2
fix: TODO related fixes for post-v20 release
2023-11-19 10:14:13 -06:00
Konstantin Akimov
a620a6b6cd refactor: a new struct CDKGJustification::Contribution instead std::pair 2023-11-18 02:43:47 +07:00
Konstantin Akimov
3f902384c9 cleanup: remove TODO so far as it is not clear what exactly to do 'cleanup' 2023-11-18 02:43:47 +07:00
Konstantin Akimov
33728107ec cleanup: drop UpgradeDB for llmq::BlockProcessor 2023-11-18 02:43:47 +07:00
Konstantin Akimov
e6391b606a refactor: use emplace_back in simplifiedmns constructors 2023-11-18 02:43:47 +07:00
Konstantin Akimov
d82b0ec72e feat: improve log of amount in governance classes 2023-11-18 02:43:47 +07:00
Konstantin Akimov
cfe9efe793 cleanup: remove outdated TODO to follow-up bitcoin#16624 2023-11-18 02:43:47 +07:00
Konstantin Akimov
7f15dcff8a refactor: avoid code duplication and data copy 2023-11-18 02:43:47 +07:00
fanquake
0beae4007c Merge bitcoin/bitcoin#23878: doc: Remove TODO comment in tx_verify
fa50d8b66e74792eb46515853e4aa8a99f25561b doc: Remove TODO comment in tx_verify (MarcoFalke)

Pull request description:

  The comment has no clear motivation, so it seems better to remove it and fix it when there is a reason.

  An alternative (if a fix isn't possible when there is a clear motivation) would be to create an issue thread for easier discussion.

ACKs for top commit:
  fanquake:
    ACK fa50d8b66e74792eb46515853e4aa8a99f25561b

Tree-SHA512: e9c25bab46a73b7c2db288c62ed9838a5e794b3b72db494173f4502da60b58dec4383064964c0842932cd30e4251fc01ad0c28681e2ef6cb442482eea2bad595
2023-11-18 02:43:42 +07:00
Konstantin Akimov
89f10e7d67 cleanup: remove out-dated TODO
TODO itself is removed in bitcoin#10618 (DNM)
But actually fixed in [merged] Merge #18021: Convert undo.h to new serialization framework
2023-11-18 02:41:04 +07:00
Konstantin Akimov
ba97f49f2f
refactor: re-order headers and forward declarations to improve compile time (#5693)
## Issue being fixed or feature implemented
Some headers include other heavy headers, such as `logging.h`,
`tinyformat.h`, `iostream`. These headers are heavy and increase
compilation time on scale of whole project drastically because can be
used in many other headers.

## What was done?
Moved many heavy includes from headers to cpp files to optimize
compilation time.
In some places  added forward declarations if it is reasonable.

As side effect removed 2 circular dependencies:
```
"llmq/debug -> llmq/dkgsessionhandler -> llmq/debug"
"llmq/debug -> llmq/dkgsessionhandler -> llmq/dkgsession -> llmq/debug"
```


## How Has This Been Tested?
Run build 2 times before refactoring and after refactoring: `make clean
&& sleep 10s; time make -j18`

Before refactoring:
```
real    5m37,826s
user    77m12,075s
sys     6m20,547s

real    5m32,626s
user    76m51,143s
sys     6m24,511s
```

After refactoring:
```
real    5m18,509s
user    73m32,133s
sys     6m21,590s

real    5m14,466s
user    73m20,942s
sys     6m17,868s
```

~5% of improvement for compilation time. That's not huge, but that's
worth to get merged

There're several more refactorings TODO but better to do them later by
backports:
 - bitcoin/bitcoin#27636
 - bitcoin/bitcoin#26286
 - bitcoin/bitcoin#27238
 - and maybe this one: bitcoin/bitcoin#28200


## Breaking Changes
N/A

## Checklist:
- [x] I have performed a self-review of my own code
- [ ] I have commented my code, particularly in hard-to-understand areas
- [ ] I have added or updated relevant unit/integration/functional/e2e
tests
- [ ] I have made corresponding changes to the documentation
- [x] I have assigned this pull request to a milestone
2023-11-17 10:04:18 -06:00
PastaPastaPasta
00f3db115d
Merge pull request #5714 from UdjinM6/bp5713
backport: backport 5713 to v20.x
2023-11-16 13:24:23 -06:00
UdjinM6
ea1f5241f9
fix: make CONFIGFLAGS optional (#5713)
## Issue being fixed or feature implemented
make it possible to run `./contrib/guix/guix-build` without specifying
`CONFIGFLAGS`

## What was done?

## How Has This Been Tested?
run `./contrib/guix/guix-build` w/ and w/out this patch

## Breaking Changes
n/a

## Checklist:
- [x] I have performed a self-review of my own code
- [ ] I have commented my code, particularly in hard-to-understand areas
- [ ] I have added or updated relevant unit/integration/functional/e2e
tests
- [ ] I have made corresponding changes to the documentation
- [x] I have assigned this pull request to a milestone _(for repository
code-owners and collaborators only)_
2023-11-16 22:21:16 +03:00
UdjinM6
7fc8a04e1c
fix: make CONFIGFLAGS optional (#5713)
## Issue being fixed or feature implemented
make it possible to run `./contrib/guix/guix-build` without specifying
`CONFIGFLAGS`

## What was done?

## How Has This Been Tested?
run `./contrib/guix/guix-build` w/ and w/out this patch

## Breaking Changes
n/a

## Checklist:
- [x] I have performed a self-review of my own code
- [ ] I have commented my code, particularly in hard-to-understand areas
- [ ] I have added or updated relevant unit/integration/functional/e2e
tests
- [ ] I have made corresponding changes to the documentation
- [x] I have assigned this pull request to a milestone _(for repository
code-owners and collaborators only)_
2023-11-16 13:19:18 -06:00
PastaPastaPasta
702bc49ac2
Merge pull request #5712 from PastaPastaPasta/20.0.1-backports
backport: v20.0.1 backports and release
2023-11-16 12:56:30 -06:00
pasta
39b315aef7
chore: bump version to 20.0.1 2023-11-16 12:42:35 -06:00
pasta
f914cdd52b
docs: archive v20.0.0 release notes and create v20.0.1 release notes 2023-11-16 12:41:31 -06:00