Commit Graph

21824 Commits

Author SHA1 Message Date
fanquake
a1ecac0f61
Merge bitcoin/bitcoin#22783: build: Cleanup depends build system
539ca409c939a31bc51f41f14ebb8bf8f48e0073 build: Remove unneeded share/man directory from libXau package (Hennadii Stepanov)
6c25c83050a8401a76502a1f0ace0ca1428e2916 build: Remove unneeded share/man directory from freetype package (Hennadii Stepanov)
9067c6c451262222a11785ce9622dd6627644cf1 build: Remove empty var/cache/fontconfig directory from fontconfig (Hennadii Stepanov)
4a37c268dbeed3a361286dcd090aea779527d996 build: Remove unneeded share/doc directory from expat package (Hennadii Stepanov)
acb9400ab602065d0996f3901de418b710a18159 build: Drop non-existent share/pkgconfig directory (Hennadii Stepanov)

Pull request description:

  This PR:
  - removes non-existent `share/pkgconfig` path from `PKG_CONFIG_PATH`. This change, actually, make `PKG_CONFIG_PATH` unused in the depends build system
  - removes `doc`,  `man` and empty directories from the built packages

ACKs for top commit:
  fanquake:
    ACK 539ca409c939a31bc51f41f14ebb8bf8f48e0073

Tree-SHA512: 41ffd5cea962f7533cb5d66ff9e8fd71a3dd7a8d9568b1bb63fc68e7070d7e416f6db02a0f8ab4d94063ee7f6370f00d62a5791b44f0d21c10666af590268c36
2022-06-27 11:37:03 -05:00
fanquake
c1969bb2e8
Merge bitcoin/bitcoin#21788: build: Silence [-Wunused-command-line-argument] warnings
e9f948c72790136656df6056fd9e3698f360e077 build: Convert warnings into errors when testing for -fstack-clash-protection (Hennadii Stepanov)

Pull request description:

  Apple clang version 12.0.5 (clang-1205.0.22.9) that is a part of Xcode 12.5, and is based on LLVM clang 11.1.0, fires spammy warnings:

  ```
  clang: warning: argument unused during compilation: '-fstack-clash-protection' [-Wunused-command-line-argument]
  ```

  From the https://github.com/apple/llvm-project:
  ```
  $ git log --oneline | grep 'stack-clash-protection'
  00065d5cbd02 Revert "-fstack-clash-protection: Return an actual error when used on unsupported OS"
  4d59c8fdb955 -fstack-clash-protection: Return an actual error when used on unsupported OS
  df3bfaa39071 [Driver] Change -fnostack-clash-protection to  -fno-stack-clash-protection
  68e07da3e5d5 [clang][PowerPC] Enable -fstack-clash-protection option for ppc64
  515bfc66eace [SystemZ] Implement -fstack-clash-protection
  e67cbac81211 Support -fstack-clash-protection for x86
  454621160066 Revert "Support -fstack-clash-protection for x86"
  0fd51a4554f5 Support -fstack-clash-protection for x86
  658495e6ecd4 Revert "Support -fstack-clash-protection for x86"
  e229017732bc Support -fstack-clash-protection for x86
  b03c3d8c6209 Revert "Support -fstack-clash-protection for x86"
  4a1a0690ad68 Support -fstack-clash-protection for x86
  f6d98429fcdb Revert "Support -fstack-clash-protection for x86"
  39f50da2a357 Support -fstack-clash-protection for x86
  ```

  I suppose, that Apple clang-1205.0.22.9 ends with on of the "Revert..." commits.

  This PR prevents using of the `-fstack-clash-protection` flag if it causes warnings.

  ---

  System: macOS Big Sur 11.3 (20E232).

ACKs for top commit:
  jarolrod:
    re-ACK e9f948c72790136656df6056fd9e3698f360e077
  Sjors:
    tACK e9f948c72790136656df6056fd9e3698f360e077 on macOS 11.3.1

Tree-SHA512: 30186da67f9b0f34418014860c766c2e7f622405520f1cbbc1095d4aa4038b0a86014d76076f318a4b1b09170a96d8167c21d7f53a760e26017f486e1a7d39d4
2022-06-27 11:37:03 -05:00
MarcoFalke
f3b7c61a9a
Merge bitcoin/bitcoin#21746: refactor: init: mark fReset const
785f9cc46a43661c63a5ec56a9e82f4ce9d42f44 refactor: init: mark fReset const (James O'Beirne)

Pull request description:

  Small thing, but hey - it doesn't change.

ACKs for top commit:
  theStack:
    Code-review ACK 785f9cc46a43661c63a5ec56a9e82f4ce9d42f44

Tree-SHA512: 3cb8d7037f517162f6315d561accc4932b0f1e340162c3283871433f2e355d57b3740c9d2e953ce33fbfa3b277c8437f91955fb70331b3fe9c8e6a8589dc2b49
2022-06-27 11:37:03 -05:00
fanquake
46cc6423b1
Merge #21311: rpc: document optional fields for getchaintxstats result
73e1f7d754c0a2381254447d692fe27a5af8c1c5 rpc: document optional fields for getchaintxstats result (Sebastian Falbesoner)

Pull request description:

  This mini-PR updates the result help of the `getchaintxstats` RPC by showing the following fields as "optional":
  - window_tx_count
  - window_interval
  - txrate

  Help output diff between master and PR branch:
  ```diff
  16,18c16,18
  <   "window_tx_count" : n,                (numeric) The number of transactions in the window. Only returned if "window_block_count" is > 0
  <   "window_interval" : n,                (numeric) The elapsed time in the window in seconds. Only returned if "window_block_count" is > 0
  <   "txrate" : n                          (numeric) The average rate of transactions per second in the window. Only returned if "window_interval" is > 0
  ---
  >   "window_tx_count" : n,                (numeric, optional) The number of transactions in the window. Only returned if "window_block_count" is > 0
  >   "window_interval" : n,                (numeric, optional) The elapsed time in the window in seconds. Only returned if "window_block_count" is > 0
  >   "txrate" : n                          (numeric, optional) The average rate of transactions per second in the window. Only returned if "window_interval" is > 0
  ```

ACKs for top commit:
  0xB10C:
    ACK 73e1f7d754c0a2381254447d692fe27a5af8c1c5

Tree-SHA512: 63c8db3e47a3c2d5564d53c564484b95b656e1e5deca1e9841bc90d122d3c81f02fd2b59313fd913ce81b16f7cc2969fe1dd9d6c3e23628b8ac057ea08f55daa
2022-06-27 11:37:03 -05:00
fanquake
b2df26bec3
Merge #21421: build: don't try and use -fstack-clash-protection on Windows
7b3434f8002d1a8cf0dbd0a0caef28e783b1efd8 build: don't try and use -fstack-clash-protection on Windows (fanquake)

Pull request description:

  This has never worked with any of the mingw-w64 compilers we use, and the `-O0` is causing issues for builders applying spectre mitigations (see [IRC logs](http://www.erisian.com.au/bitcoin-core-dev/log-2021-03-12.html#l-15)).

  Recent discussion on https://gcc.gnu.org/bugzilla/show_bug.cgi?id=90458 would also indicate that this should just not be used on Windows.

ACKs for top commit:
  laanwj:
    Concept and code review ACK (but untested) 7b3434f8002d1a8cf0dbd0a0caef28e783b1efd8
  hebasto:
    ACK 7b3434f8002d1a8cf0dbd0a0caef28e783b1efd8, I've verified that this change does not affect builds for `HOST=x86_64-w64-mingw32` by comparing sizes of the output `*.exe` files.

Tree-SHA512: 72b582321ddff8db3201460fa42a53304e70f141ae078d76a4d4eeb1ca27c8dd567ccb468cc8616179c8df784bd8ca038dcb9a277f9e29f9d98c3cc842916b18
2022-06-27 11:37:02 -05:00
MarcoFalke
10979f1a74
Merge #21317: util: Make Assume() usable as unary expression
fa4cebadcffd9112da4b13c7cc7ccf21e2bee887 util: Make Assume() usable as unary expression (MarcoFalke)

Pull request description:

  Assume shouldn't behave different at the call site depending on build flags. Currently compilation fails if it is used as expression. Fix that by using the lambda approach from `Assert()` without the `assert()`.

ACKs for top commit:
  jnewbery:
    ACK fa4cebadcffd9112da4b13c7cc7ccf21e2bee887
  practicalswift:
    cr ACK fa4cebadcffd9112da4b13c7cc7ccf21e2bee887: patch looks correct and commit hash starts with `fa`

Tree-SHA512: 9ec9ac8d410cdaf5e4e28df571a89e3d23d38e05a7027bb726cae3da6e9314734277e5a218e9e090cc17e10db763da71052c229ad642077ca5824ee42022f3ed
2022-06-27 11:37:02 -05:00
fanquake
b50457044c
Merge #20720: build: more robustly check for fcf-protection support
e9189a750b237eba1befc6b16c12c2cee3e0176c build: more robustly check for fcf-protection support (fanquake)

Pull request description:

  When using Clang 7, we may end up trying to use the flag when it won't
  work properly, which can lead to confusing errors. i.e:
  ```bash
  /usr/bin/ld: error: ... <corrupt x86 feature size: 0x8>
  ```

  Use `AX_CHECK_LINK_FLAG` & `--fatal-warnings` to ensure we wont use the flag in this case.

  We do this as even when the error is emitted, compilation succeeds, and the binaries produced will run. This means we can't just check if the compiler accepts the flag, or if compilation succeeds (without or without `-Werror`, and/or passing `-Wl,--fatal-warnings`, which may not be passed through to the linker).

  This was reported by someone configuring for fuzzing, on Debian 10, where Clang 7 is the default.

  See here for a minimal example of the problematic behaviour:
  https://gist.github.com/fanquake/9b33555fcfebef8eb8c0795a71732bc6

ACKs for top commit:
  pstratem:
    tested ACK e9189a750b237eba1befc6b16c12c2cee3e0176c
  MarcoFalke:
    not an ACK e9189a750b237eba1befc6b16c12c2cee3e0176c , I only tested configure on my system (gcc-10, clang-11):
  hebasto:
    ACK e9189a750b237eba1befc6b16c12c2cee3e0176c, tested with clang-7, clang-10 and gcc: the `-fcf-protection=full` is not applied for clang-7, but applied for others compilers.

Tree-SHA512: ec24b0cc5523b90139c96cbb33bb98d1e6a24d858c466aa7dfb3c474caf8c50aca53e570fdbc0ff88378406b0ac5d687542452637b1b5fa062e829291b886fc1
2022-06-18 22:14:04 -07:00
Jonas Schnelli
2622653e74
Merge #20563: build: Check that Homebrew's berkeley-db4 package is actually installed
d3ef947524a07f8d7fbad5b95781ab6cacb1cb49 build: Check that Homebrew's berkeley-db4 package is actually installed (Hennadii Stepanov)

Pull request description:

  On master (a0489f3472f3799dc1ece32a59556fd239c4c14b) the `configure` script is not able to determine that Homebrew's `berkeley-db4` package is uninstalled. This causes a compile error on macOS.

  With this PR, and with the [uninstalled](https://stackoverflow.com/questions/20802320/detect-if-homebrew-package-is-installed) `berkeley-db4` package:
  ```
  % ./configure -q
  configure: error: libdb_cxx headers missing, Bitcoin Core requires this library for BDB wallet support (--without-bdb to disable BDB wallet support)
  ```

  Related #20478.

ACKs for top commit:
  promag:
    Tested ACK d3ef947524a07f8d7fbad5b95781ab6cacb1cb49.
  willcl-ark:
    tACK d3ef947524a07f8d7fbad5b95781ab6cacb1cb49
  jonasschnelli:
    utACK d3ef947524a07f8d7fbad5b95781ab6cacb1cb49

Tree-SHA512: 8dc532e08249ec63bd357594aa458d314b6e8537fc63f5b1d509c84d0d71d5b1f70172caa1a7efe2fc8af31c829e7982a0695cf3fbe5cbc477019550269915e1
2022-06-18 22:14:04 -07:00
Jonas Schnelli
6d5109e90f
Merge #20478: Don't set BDB flags when configuring without
982e548a9a78b1b0abad59b54c780b6b06570452 Don't set BDB flags when configuring without (Jonas Schnelli)

Pull request description:

  Configuring `--without-bdb` on MacOS leads to a compile error (when BerkeleyDB is not installed). `brew --prefix berkeley-db4` always reports the target directory (even if not installed).

  This PR prevents BDB_CFLAGS (et al) from being populated when configuring `--without-bdb`

  ```
  ld: warning: directory not found for option '-L/Users/user/Documents/homebrew/Cellar/berkeley-db@4/4.8.30/lib'
  ld: warning: directory not found for option '-L/Users/user/Documents/homebrew/Cellar/berkeley-db@4/4.8.30/lib'
  ld: library not found for -ldb_cxx-4.8
  ld: library not found for -ldb_cxx-4.8
  ```

ACKs for top commit:
  promag:
    Tested ACK 982e548a9a78b1b0abad59b54c780b6b06570452.
  hebasto:
    ACK 982e548a9a78b1b0abad59b54c780b6b06570452, tested on macOS 11 Big Sur.

Tree-SHA512: f8ca0adca0e18e2de4c0f99d5332cba70d957a9d31a357483b43dcf61c2ed4749d223eabadd45fdbf3ef0781c6b37217770e9aa935b5207eaf7f87c5bdfe9e95
2022-06-18 22:14:04 -07:00
fanquake
1ce15a1ac6
Merge #20430: sanitizers: Add suppression for unsigned-integer-overflow in libstdc++
0f020cdf0a8c3c88499c96522470e2e5e79f27fa sanitizers: Add suppression for unsigned-integer-overflow in libstdc++ basic_string.tcc (Jonas Schnelli)

Pull request description:

  Reported here: https://bitcoinbuilds.org/logs/e35cd579-0f0f-47e4-b49a-4ceba8ff9830.log
  Issue: https://github.com/gcc-mirror/gcc/blob/master/libstdc%2B%2B-v3/include/bits/basic_string.tcc#L1271

ACKs for top commit:
  MarcoFalke:
    cr ACK 0f020cdf0a8c3c88499c96522470e2e5e79f27fa
  practicalswift:
    cr ACK 0f020cdf0a8c3c88499c96522470e2e5e79f27fa

Tree-SHA512: e304259a1eed878263bd715b4d16c57f8974264c23ccd6799f85e8141b2eb0b5c6468a6452ffbc7334f57c1957b6e43bb248760b3c0718d93f092d57764d0a8f
2022-06-18 22:14:04 -07:00
Wladimir J. van der Laan
e1b091be3b
Merge #18921: build: add stack-clash and control-flow protection options to hardening flags
b536813cefc13f5c54a28a7c2fce8c69e89d6624 build: add -fstack-clash-protection to hardening flags (fanquake)
076183b36b76a11438463883ff916f17aef9e001 build: add -fcf-protection=full to hardening options (fanquake)

Pull request description:

  Beginning with Ubuntu `19.10`, it's packaged GCC now has some additional hardening options enabled by default (in addition to existing defaults like `-fstack-protector-strong` and reducing the minimum ssp buffer size). The new additions are`-fcf-protection=full` and `-fstack-clash-protection`.

  > -fcf-protection=[full|branch|return|none]
  > Enable code instrumentation of control-flow transfers to increase program security by checking that target addresses of control-flow transfer instructions (such as indirect function call, function return, indirect jump) are valid. This prevents diverting the flow of control to an unexpected target. This is intended to protect against such threats as Return-oriented Programming (ROP), and similarly call/jmp-oriented programming (COP/JOP).

  > -fstack-clash-protection
  > Generate code to prevent stack clash style attacks. When this option is enabled, the compiler will only allocate one page of stack space at a time and each page is accessed immediately after allocation. Thus, it prevents allocations from jumping over any stack guard page provided by the operating system.

  If your interested you can grab `gcc-9_9.3.0-10ubuntu2.debian.tar.xz` from https://packages.ubuntu.com/focal/g++-9. The relevant changes are part of the `gcc-distro-specs` patches, along with the relevant additions to the gcc manages:

  > NOTE: In Ubuntu 19.10 and later versions, -fcf-protection is enabled by default for C, C++, ObjC, ObjC++, if none of -fno-cf-protection nor -fcf-protection=* are found.

  > NOTE: In Ubuntu 19.10 and later versions, -fstack-clash-protection is enabled by default for C, C++, ObjC, ObjC++, unless -fno-stack-clash-protection is found.

  So, if you're C++ using GCC on Ubuntu 19.10 or later, these options will be active unless you explicitly opt out. This can be observed with a small test:

  ```c++
  int main() { return 0; }
  ```

  ```bash
  g++ --version
  g++ (Ubuntu 9.3.0-10ubuntu2) 9.3.0

  g++ test.cpp

  objdump -dC a.out
  ..
  0000000000001129 <main>:
      1129:	f3 0f 1e fa          	endbr64
      112d:	55                   	push   %rbp
      112e:	48 89 e5             	mov    %rsp,%rbp
      1131:	b8 00 00 00 00       	mov    $0x0,%eax
      1136:	5d                   	pop    %rbp
      1137:	c3                   	retq
      1138:	0f 1f 84 00 00 00 00 	nopl   0x0(%rax,%rax,1)
      113f:	00

  # recompile opting out of control flow protection
  g++ test.cpp -fcf-protection=none

  objdump -dC a.out
  ...
  0000000000001129 <main>:
      1129:	55                   	push   %rbp
      112a:	48 89 e5             	mov    %rsp,%rbp
      112d:	b8 00 00 00 00       	mov    $0x0,%eax
      1132:	5d                   	pop    %rbp
      1133:	c3                   	retq
      1134:	66 2e 0f 1f 84 00 00 	nopw   %cs:0x0(%rax,%rax,1)
      113b:	00 00 00
      113e:	66 90                	xchg   %ax,%ax
  ```

  Note the insertion of an `endbr64` instruction when compiling and _not_ opting out. This instruction is part of the Intel Control-flow Enforcement Technology [spec](https://software.intel.com/sites/default/files/managed/4d/2a/control-flow-enforcement-technology-preview.pdf), which the GCC control flow implementation is based on.

  If we're still doing gitian builds for the `0.21.0` and `0.22.0` releases, we'd likely update the gitian image to Ubuntu Focal, which would mean that the GCC used for gitian builds would also be using these options by default. So we should decide whether we want to explicitly turn these options on as part of our hardening options (although not just for this reason), or, we should be opting-out.

  GCC has supported both options since 8.0.0. Clang has supported `-fcf-protection` from 7.0.0 and will support `-fstack-clash-protection` in it's upcoming [11.0.0 release](https://clang.llvm.org/docs/ReleaseNotes.html#id6).

ACKs for top commit:
  jamesob:
    ACK b536813cefc13f5c54a28a7c2fce8c69e89d6624 ([`jamesob/ackr/18921.1.fanquake.build_add_stack_clash_an`](https://github.com/jamesob/bitcoin/tree/ackr/18921.1.fanquake.build_add_stack_clash_an))
  laanwj:
    Code review ACK b536813cefc13f5c54a28a7c2fce8c69e89d6624

Tree-SHA512: abc9adf23cdf1be384f5fb9aa5bfffdda86b9ecd671064298d4cda0440828b509f070f9b19c88c7ce50ead9ff32afff9f14c5e78d75f01241568fbfa077be0b7
2022-06-18 22:14:03 -07:00
fanquake
37d58e402c
Merge #19356: build: Fix search for brew-installed BDB 4 on OS X
8578c6fccd11404412d2c60f9bede311b79ca0d0 build: Fix search for brew-installed BDB 4 on OS X (Glenn Willen)

Pull request description:

  ~~NOTE: This PR contains one important fix that I need (to make Bitcoin Core build cleanly on my system without shenanigans), plus some related general cleanup that is not really necessary, and could be annoying. (I am prepared to defend my argument that BDB_CFLAGS is wrong here, and BDB_CPPFLAGS is right, but this could bite anybody who has gotten in the habit of -- or scripted -- setting the former.)~~

  Ok, I have been convinced that I was too clever with the refactor and I have removed it. Now it's just the tiny change to fix the build on my local machine.

  ---

  On OS X, when searching Homebrew keg-only packages for BDB 4.8, if we find it,
  use BDB_CPPFLAGS and BDB_LIBS instead of CFLAGS and LIBS for the result. This
  is (1) more correct, and (2) necessary in order to give this location
  priority over other directories in the include search path, which may include
  system include directories with other versions of BDB.

ACKs for top commit:
  theuni:
    ACK 8578c6fccd11404412d2c60f9bede311b79ca0d0.

Tree-SHA512: a28f48fc81a25736f7e77c663f21cd9a6ae1cd115682031c5aa695c94cb5afa11920330a60cd6a54832822a2aec1eb23123ac2e2dcd4f0b3835aef9c9339ac97
2022-06-18 22:14:03 -07:00
Wladimir J. van der Laan
33c473a8e3
Merge #18796: scripts: security-check.py refactors
eacedfb0230978748cbcfb13817fed7e7c756ba7 scripts: add additional type annotations to security-check.py (fanquake)
83d063e9541cc9ea41ea86919eb9435c73efb14e scripts: add run_command to security-check.py (fanquake)
13f606b4f940e5820ff21ea62fc27a5a91774b05 scripts: remove NONFATAL from security-check.py (fanquake)
061acf62a15ad3dbb9f055b7c2569b9832ed623a scripts: no-longer check for 32 bit windows in security-check.py (fanquake)

Pull request description:

  * Remove 32-bit Windows checks.
  * Remove NONFATAL checking. Added in #8249, however unused since #13764.
  * Add `run_command` to de-duplicate all of the subprocess calls. Mentioned in #18713.
  * Add additional type annotations.
  * Print stderr when there is an issue running a command.

ACKs for top commit:
  laanwj:
    ACK eacedfb0230978748cbcfb13817fed7e7c756ba7

Tree-SHA512: 69a7ccfdf346ee202b3e8f940634c5daed1d2b5a5d15ac9800252866ba3284ec66e391a66a0b341f5a4e5e8482fe1b614d4671e8e766112ff059405081184a85
2022-06-18 22:14:03 -07:00
fanquake
f7a94b58d4
Merge #18629: scripts: add PE .reloc section check to security-check.py
3e38023af724a76972d39cbccfb0bba4c54a0323 scripts: add PE .reloc section check to security-check.py (fanquake)

Pull request description:

  The `ld` in binutils has historically had a few issues with PE binaries, there's a good summary in this [thread](https://sourceware.org/bugzilla/show_bug.cgi?id=19011).

  One issue in particular was `ld` stripping the `.reloc` section out of PE binaries, even though it's required for functioning ASLR. This was [reported by a Tor developer in 2014](https://sourceware.org/bugzilla/show_bug.cgi?id=17321) and they have been patching their [own binutils](https://gitweb.torproject.org/builders/tor-browser-build.git/tree/projects/binutils) ever since. However their patch only made it into binutils at the [start of this year](https://sourceware.org/git/gitweb.cgi?p=binutils-gdb.git;a=commit;h=dc9bd8c92af67947db44b3cb428c050259b15cd0). It adds an `--enable-reloc-section` flag, which is turned on by default if you are using `--dynamic-base`. In the mean time this issue has also been worked around by other projects, such as FFmpeg, see [this commit](91b668acd6).

  I have checked our recent supported Windows release binaries, and they do contain a `.reloc` section. From what I understand, we are using all the right compile/linker flags, including `-pie` & `-fPIE`, and have never run into the crashing/entrypoint issues that other projects might have seen.

  One other thing worth noting here, it how Debian/Ubuntu patch the binutils that they distribute, because that's what we end up using in our gitian builds.

  In the binutils-mingw-w64 in Bionic (18.04), which we currently use in gitian, PE hardening options/security flags are enabled by default. See the [changelog](https://changelogs.ubuntu.com/changelogs/pool/universe/b/binutils-mingw-w64/binutils-mingw-w64_8ubuntu1/changelog) and the [relevant commit](452b3013b8).

  However in Focal (20.04), this has now been reversed. PE hardening options are no-longer the default. See the [changelog](https://changelogs.ubuntu.com/changelogs/pool/universe/b/binutils-mingw-w64/binutils-mingw-w64_8.8/changelog) and [relevant commit](7bd8b2fbc2), which cites same .reloc issue mentioned here.

  Given that we explicitly specify/opt-in to everything that we want to use, the defaults aren't necessarily an issue for us. However I think it highlights the importance of continuing to be explicit about what we want, and not falling-back or relying on upstream.

  This was also prompted by the possibility of us doing link time garbage collection, see #18579 & #18605. It seemed some sanity checks would be worthwhile in-case the linker goes haywire while garbage collecting.

  I think Guix is going to bring great benefits when dealing with these kinds of issues. Carl you might have something to say in that regard.

ACKs for top commit:
  dongcarl:
    ACK 3e38023af724a76972d39cbccfb0bba4c54a0323

Tree-SHA512: af14d63bdb334bde548dd7de3e0946556b7e2598d817b56eb4e75b3f56c705c26aa85dd9783134c4b6a7aeb7cb4de567eed996e94d533d31511f57ed332287da
2022-06-18 22:14:03 -07:00
fanquake
bd7b5a3ad4
Merge #18713: scripts: Add MACHO stack canary check to security-check.py
8334ee31f868f0f9baf0920d14d20174ed889dbe scripts: add MACHO LAZY_BINDINGS test to test-security-check.py (fanquake)
7b99c7454cdb74cd9cd7a5eedc2fb9d0a19df456 scripts: add MACHO Canary check to security-check.py (fanquake)

Pull request description:

  7b99c7454cdb74cd9cd7a5eedc2fb9d0a19df456 uses `otool -Iv` to check for `___stack_chk_fail` in the macOS binaries. Similar to the [ELF check](https://github.com/bitcoin/bitcoin/blob/master/contrib/devtools/security-check.py#L105). Note that looking for a triple underscore prefixed function (as opposed to two for ELF) is correct for the macOS binaries. i.e:
  ```bash
  otool -Iv bitcoind | grep chk
  0x00000001006715b8   509 ___memcpy_chk
  0x00000001006715be   510 ___snprintf_chk
  0x00000001006715c4   511 ___sprintf_chk
  0x00000001006715ca   512 ___stack_chk_fail
  0x00000001006715d6   517 ___vsnprintf_chk
  0x0000000100787898   513 ___stack_chk_guard
  ```

  8334ee31f868f0f9baf0920d14d20174ed889dbe is a follow up to #18295 and adds test cases to `test-security-check.py` that for some reason I didn't add at the time. I'll sort out #18434 so that we can run these tests in the CI.

ACKs for top commit:
  practicalswift:
    ACK 8334ee31f868f0f9baf0920d14d20174ed889dbe: Mitigations are important. Important things are worth asserting :)
  jonasschnelli:
    utACK 8334ee31f868f0f9baf0920d14d20174ed889dbe.

Tree-SHA512: 1aa5ded34bbd187eddb112b27278deb328bfc21ac82316b20fab6ad894f223b239a76b53dab0ac1770d194c1760fcc40d4da91ec09959ba4fc8eadedb173936a
2022-06-18 22:14:03 -07:00
fanquake
96866c4579
Merge #18709: doc: note why we can't use thread_local with glibc back compat
b155fcda5186c59fc4fb2a9eaaf791d132e0ab30 doc: fix typo in configure.ac (fanquake)
20a30922fbf6ba14e250ca649239af115dbbe7b0 doc: note why we can't use thread_local with glibc back compat (fanquake)

Pull request description:

  Given that we went through a [gitian build](https://github.com/bitcoin/bitcoin/pull/18681) to remember why this is the case, we might as well make a note of it in configure.ac.

  [From #18681](https://github.com/bitcoin/bitcoin/pull/18681#issuecomment-615526634):

  Looking at the Linux build log, this has failed with:
  ```bash
  Checking glibc back compat...
  bitcoind: symbol __cxa_thread_atexit_impl from unsupported version GLIBC_2.18
  bitcoind: failed IMPORTED_SYMBOLS
  bitcoin-cli: symbol __cxa_thread_atexit_impl from unsupported version GLIBC_2.18
  bitcoin-cli: failed IMPORTED_SYMBOLS
  bitcoin-tx: symbol __cxa_thread_atexit_impl from unsupported version GLIBC_2.18
  bitcoin-tx: failed IMPORTED_SYMBOLS
  bitcoin-wallet: symbol __cxa_thread_atexit_impl from unsupported version GLIBC_2.18
  bitcoin-wallet: failed IMPORTED_SYMBOLS
  test/test_bitcoin: symbol __cxa_thread_atexit_impl from unsupported version GLIBC_2.18
  test/test_bitcoin: failed IMPORTED_SYMBOLS
  bench/bench_bitcoin: symbol __cxa_thread_atexit_impl from unsupported version GLIBC_2.18
  bench/bench_bitcoin: failed IMPORTED_SYMBOLS
  qt/bitcoin-qt: symbol __cxa_thread_atexit_impl from unsupported version GLIBC_2.18
  ```

  `__cxa_thread_atexit_impl` is used for [thread_local variable destruction](https://sourceware.org/glibc/wiki/Destructor%20support%20for%20thread_local%20variables):

  > To implement this support, glibc defines __cxa_thread_atexit_impl exclusively for use by libstdc++ (which has the __cxa_thread_atexit to wrap around it), that registers destructors for thread_local variables in a list. Upon thread or process exit, the destructors are called in reverse order in which they were added.

  As suggested, this only became available in glibc 2.18. From the [2.18 release notes](https://sourceware.org/legacy-ml/libc-alpha/2013-08/msg00160.html):

  > * Add support for calling C++11 thread_local object destructors on thread
    and program exit.  This needs compiler support for offloading C++11
    destructor calls to glibc.

ACKs for top commit:
  hebasto:
    ACK b155fcda5186c59fc4fb2a9eaaf791d132e0ab30

Tree-SHA512: 5b9567e4a70598a4b0b91956f44ae0d93091db17c84cbf9817dac6cfa992c97d3438a8b1bb66644c74891f2149e44984daed445d22de93ca8858c5b0eabefb40
2022-06-18 22:14:02 -07:00
fanquake
67ceda1b5a
Merge #18295: scripts: add MACHO lazy bindings check to security-check.py
5ca90f8b598978437340bb8467f527b9edfb2bbf scripts: add MACHO lazy bindings check to security-check.py (fanquake)

Pull request description:

  This is a slightly belated follow up to #17686 and some discussion with Cory. It's not entirely clear if we should make this change due to the way the macOS dynamic loader appears to work. However I'm opening this for some discussion. Also related to #17768.

  #### Issue:
  [`LD64`](https://opensource.apple.com/source/ld64/) doesn't set the [MH_BINDATLOAD](https://opensource.apple.com/source/xnu/xnu-6153.11.26/EXTERNAL_HEADERS/mach-o/loader.h.auto.html) bit in the header of MACHO executables, when building with `-bind_at_load`. This is in contradiction to the [documentation](https://opensource.apple.com/source/ld64/ld64-450.3/doc/man/man1/ld.1.auto.html):
  ```bash
  -bind_at_load
       Sets a bit in the mach header of the resulting binary which tells dyld to
       bind all symbols when the binary is loaded, rather than lazily.
  ```

  The [`ld` in Apples cctools](https://opensource.apple.com/source/cctools/cctools-927.0.2/ld/layout.c.auto.html) does set the bit, however the [cctools-port](https://github.com/tpoechtrager/cctools-port/) that we use for release builds, bundles `LD64`.

  However; even if the linker hasn't set that bit, the dynamic loader ([`dyld`](https://opensource.apple.com/source/dyld/)) doesn't seem to ever check for it, and from what I understand, it looks at a different part of the header when determining whether to lazily load symbols.

  Note that our release binaries are currently working as expected, and no lazy loading occurs.

  #### Example:

  Using a small program, we can observe the behaviour of the dynamic loader.

  Conducted using:
  ```bash
  clang++ --version
  Apple clang version 11.0.0 (clang-1100.0.33.17)
  Target: x86_64-apple-darwin18.7.0

  ld -v
  @(#)PROGRAM:ld  PROJECT:ld64-530
  BUILD 18:57:17 Dec 13 2019
  LTO support using: LLVM version 11.0.0, (clang-1100.0.33.17) (static support for 23, runtime is 23)
  TAPI support using: Apple TAPI version 11.0.0 (tapi-1100.0.11)
  ```

  ```cpp
  #include <iostream>
  int main() {
  	std::cout << "Hello World!\n";
  	return 0;
  }
  ```

  Compile and check the MACHO header:
  ```bash
  clang++ test.cpp -o test
  otool -vh test
  ...
  Mach header
        magic cputype cpusubtype  caps    filetype ncmds sizeofcmds      flags
  MH_MAGIC_64  X86_64        ALL LIB64     EXECUTE    16       1424   NOUNDEFS DYLDLINK TWOLEVEL WEAK_DEFINES BINDS_TO_WEAK PIE

  # Run and dump dynamic loader bindings:
  DYLD_PRINT_BINDINGS=1 DYLD_PRINT_TO_FILE=no_bind.txt ./test
  Hello World!
  ```

  Recompile with `-bind_at_load`. Note still no `BINDATLOAD` flag:
  ```bash
  clang++ test.cpp -o test -Wl,-bind_at_load
  otool -vh test
  Mach header
        magic cputype cpusubtype  caps    filetype ncmds sizeofcmds      flags
  MH_MAGIC_64  X86_64        ALL LIB64     EXECUTE    16       1424   NOUNDEFS DYLDLINK TWOLEVEL WEAK_DEFINES BINDS_TO_WEAK PIE
  ...
  DYLD_PRINT_BINDINGS=1 DYLD_PRINT_TO_FILE=bind.txt ./test
  Hello World!
  ```

  If we diff the outputs, you can see that `dyld` doesn't perform any lazy bindings when the binary is compiled with `-bind_at_load`, even if the `BINDATLOAD` flag is not set:
  ```diff
  @@ -1,11 +1,27 @@
  +dyld: bind: test:0x103EDF030 = libc++.1.dylib:__ZNKSt3__16locale9use_facetERNS0_2idE, *0x103EDF030 = 0x7FFF70C9FA58
  +dyld: bind: test:0x103EDF038 = libc++.1.dylib:__ZNKSt3__18ios_base6getlocEv, *0x103EDF038 = 0x7FFF70CA12C2
  +dyld: bind: test:0x103EDF068 = libc++.1.dylib:__ZNSt3__113basic_ostreamIcNS_11char_traitsIcEEE6sentryC1ERS3_, *0x103EDF068 = 0x7FFF70CA12B6
  +dyld: bind: test:0x103EDF070 = libc++.1.dylib:__ZNSt3__113basic_ostreamIcNS_11char_traitsIcEEE6sentryD1Ev, *0x103EDF070 = 0x7FFF70CA1528
  +dyld: bind: test:0x103EDF080 = libc++.1.dylib:__ZNSt3__16localeD1Ev, *0x103EDF080 = 0x7FFF70C9FAE6
  <trim>
  -dyld: lazy bind: test:0x10D4AC0C8 = libsystem_platform.dylib:_strlen, *0x10D4AC0C8 = 0x7FFF73C5C6E0
  -dyld: lazy bind: test:0x10D4AC068 = libc++.1.dylib:__ZNSt3__113basic_ostreamIcNS_11char_traitsIcEEE6sentryC1ERS3_, *0x10D4AC068 = 0x7FFF70CA12B6
  -dyld: lazy bind: test:0x10D4AC038 = libc++.1.dylib:__ZNKSt3__18ios_base6getlocEv, *0x10D4AC038 = 0x7FFF70CA12C2
  -dyld: lazy bind: test:0x10D4AC030 = libc++.1.dylib:__ZNKSt3__16locale9use_facetERNS0_2idE, *0x10D4AC030 = 0x7FFF70C9FA58
  -dyld: lazy bind: test:0x10D4AC080 = libc++.1.dylib:__ZNSt3__16localeD1Ev, *0x10D4AC080 = 0x7FFF70C9FAE6
  -dyld: lazy bind: test:0x10D4AC070 = libc++.1.dylib:__ZNSt3__113basic_ostreamIcNS_11char_traitsIcEEE6sentryD1Ev, *0x10D4AC070 = 0x7FFF70CA1528
  ```

  Note: `dyld` also has a `DYLD_BIND_AT_LAUNCH=1` environment variable, that when set, will force any lazy bindings to be non-lazy:
  ```bash
  dyld: forced lazy bind: test:0x10BEC8068 = libc++.1.dylib:__ZNSt3__113basic_ostream
  ```

  #### Thoughts:
  After looking at the dyld source, I can't find any checks for `MH_BINDATLOAD`. You can see the flags it does check for, such as MH_PIE or MH_BIND_TO_WEAK [here](https://opensource.apple.com/source/dyld/dyld-732.8/src/ImageLoaderMachO.cpp.auto.html).

  It seems that the lazy binding of any symbols depends on whether or not [lazy_bind_size](https://opensource.apple.com/source/xnu/xnu-6153.11.26/EXTERNAL_HEADERS/mach-o/loader.h.auto.html) from the `LC_DYLD_INFO_ONLY` load command is > 0. Which was mentioned in [#17686](https://github.com/bitcoin/bitcoin/pull/17686#issue-350216254).

  #### Changes:
  This PR is one of [Corys commits](7b6ba26178), that I've rebased and modified to make build. I've also included an addition to the `security-check.py` script to check for the flag.

  However, given the above, I'm not entirely sure this patch is the correct approach. If the linker no-longer inserts it, and the dynamic loader doesn't look for it, there might be little benefit to setting it. Or, maybe this is an oversight from Apple and needs some upstream discussion. Looking for some thoughts / Concept ACK/NACK.

  One alternate approach we could take is to drop the patch and modify security-check.py to look for `lazy_bind_size` == 0 in the `LC_DYLD_INFO_ONLY` load command, using `otool -l`.

ACKs for top commit:
  theuni:
    ACK 5ca90f8b598978437340bb8467f527b9edfb2bbf

Tree-SHA512: 444022ea9d19ed74dd06dc2ab3857a9c23fbc2f6475364e8552d761b712d684b3a7114d144f20de42328d1a99403b48667ba96885121392affb2e05b834b6e1c
2022-06-18 22:14:02 -07:00
Wladimir J. van der Laan
3c4982205c
Merge #18415: scripts: add MACHO tests to test-security-check.py
7142d50ac33e0ad7d24e49e04c1fc7e3e769ed46 scripts: rename test_64bit_PE to test_PE (fanquake)
edaca2dd123cef958699c07ab248cf0ffc71af07 scripts: add MACHO NX check to security-check.py (fanquake)
1a4e9f32efcc5f6a74290446dc58784fd85c7b31 scripts: add MACHO tests to test-security-check.py (fanquake)

Pull request description:

  Adds tests for the MACHO checks in security-check.py:
  ac579ada7e/contrib/devtools/security-check.py (L212-L214)

  I'm planning on following up with more checks in security-check.py, and corresponding tests in test-security-check.py.

  Note that you'll probably have to be on macOS to run them. You can run just this suite with `python3 test-security-check.py TestSecurityChecks.test_MACHO`.

ACKs for top commit:
  laanwj:
    ACK 7142d50ac33e0ad7d24e49e04c1fc7e3e769ed46

Tree-SHA512: ace3ca9f6df5d4fedd5988938fb7dc7563ec7dc587aa275f780b5f51e9b8d7d6f7768e0a1e05ce438510a07b8640aba92c76847b30c2990f46c66b78a0acf960
2022-06-18 22:14:02 -07:00
Wladimir J. van der Laan
3ceb45fdb7
Merge #18135: build: add --enable-determinism configure flag
3d9b41ecc02a69daa19df2bc850d60bf9917b724 build: add --enable-determinism configure flag (fanquake)

Pull request description:

  This adds a `--enable-determinsm` configure flag, which if used, will enable additional compile / link time flags to make subsequent builds of bitcoind deterministic.

  The first flag enabled is `--no-insert-timestamp`. This prevents the linker from embedding timestamps, and makes consecutive builds of `bitcoind.exe` deterministic. This will likely also be used for [Guix Windows builds](https://github.com/bitcoin/bitcoin/pull/17595#issuecomment-582696467).

  [`--no-insert-timestamp`](https://manpages.debian.org/buster/binutils-mingw-w64-x86-64/x86_64-w64-mingw32-ld.bfd.1.en.html):
  > The option --no-insert-timestamp can be used to insert a zero value for the timestamp, this ensuring that binaries produced from identical sources will compare identically.

  Diff of consecutive builds of [master](2bdc476d4d):
  ```diff
  --- bitcoind.exe.1
  +++ bitcoind.exe.2
  @@ -2,20 +2,20 @@
   00000060: 7420 6265 2072 756e 2069 6e20 444f 5320  t be run in DOS
   00000070: 6d6f 6465 2e0d 0d0a 2400 0000 0000 0000  mode....$.......
  -00000080: 5045 0000 6486 1400 57e8 445e 00da 6900  PE..d...W.D^..i.
  +00000080: 5045 0000 6486 1400 e3e9 445e 00da 6900  PE..d.....D^..i.
   00000090: e015 0100 f000 2600 0b02 021f 00de 4900  ......&.......I.
   000000a0: 00b0 5b00 008a 0000 e014 0000 0010 0000  ..[.............
   000000b0: 0000 4000 0000 0000 0010 0000 0002 0000  ..@.............
   000000c0: 0400 0000 0000 0000 0500 0200 0000 0000  ................
  -000000d0: 00f0 6a00 0006 0000 bd31 af00 0300 6001  ..j......1....`.
  +000000d0: 00f0 6a00 0006 0000 d434 af00 0300 6001  ..j......4....`.
   000000e0: 0000 2000 0000 0000 0010 0000 0000 0000  .. .............
  @@ -373594,15 +373594,15 @@
   005b35f0: 0000 0000 0000 0000 0000 0000 0000 0000  ................
  -005b3600: 0000 0000 57e8 445e 0000 0000 7ce1 5b00  ....W.D^....|.[.
  +005b3600: 0000 0000 e2e9 445e 0000 0000 7ce1 5b00  ......D^....|.[.
   005b3610: 0100 0000 2200 0000 2200 0000 28e0 5b00  ...."..."...(.[.
  ```

ACKs for top commit:
  practicalswift:
    ACK 3d9b41ecc02a69daa19df2bc850d60bf9917b724 -- patch looks correct
  laanwj:
    ACK 3d9b41ecc02a69daa19df2bc850d60bf9917b724

Tree-SHA512: 1ff9dab7fa818b1fc6b0eb3b7a1e0468aac9e2578f4451aa300a648f883fa83f83722067f1adf27ebad54157790857d1501d573adbd7ebdf6962858cc669960d
2022-06-18 22:14:02 -07:00
Wladimir J. van der Laan
091865b02c
Merge #18082: logging: enable thread_local usage on macOS
d76894987d0277e8011932ab7dfd77c537f8ea6e logging: enable thread_local usage on macOS (fanquake)

Pull request description:

  Now that we're building against a newer SDK (`10.14`), we should be able to enable `thread_local` usage on macOS. Have tested building and running locally, as well as cross-compiling and running the binaries on a macOS 10.14 system.

  #### master 8a56f79d491271120abc3843c46e9dda44edd308
  ```bash
  src/bitcoind -logthreadnames=1
  2020-02-06T04:38:33Z [] Bitcoin Core version v0.19.99.0-8a56f79d4 (release build)
  2020-02-06T04:38:33Z [] Assuming ancestors of block 00000000000000000005f8920febd3925f8272a6a71237563d78c2edfdd09ddf have valid signatures.
  2020-02-06T04:38:33Z [] Setting nMinimumChainWork=000000000000000000000000000000000000000008ea3cf107ae0dec57f03fe8
  2020-02-06T04:38:33Z [] Using the 'sse4(1way),sse41(4way),avx2(8way)' SHA256 implementation
  2020-02-06T04:38:33Z [] Using RdSeed as additional entropy source
  ```

  #### this PR d76894987d0277e8011932ab7dfd77c537f8ea6e
  ```bash
  checking for thread_local support... yes
  ...
  src/bitcoind -logthreadnames=1
  2020-02-06T04:17:49Z [net] net thread start
  2020-02-06T04:17:49Z [opencon] opencon thread start
  2020-02-06T04:17:49Z [dnsseed] dnsseed thread start
  2020-02-06T04:17:49Z [init] init message: Done loading
  2020-02-06T04:17:49Z [msghand] msghand thread start
  2020-02-06T04:17:49Z [addcon] addcon thread start
  ...
  2020-02-06T04:17:54Z [init] tor: Thread interrupt
  2020-02-06T04:17:54Z [init] Shutdown: In progress...
  ```

  From the [Xcode 8 release notes](https://developer.apple.com/library/archive/releasenotes/DeveloperTools/RN-Xcode/Chapters/Introduction.html#//apple_ref/doc/uid/TP40001051-CH1-SW78)
  > C++ now supports the thread_local keyword, which declares thread-local storage (TLS) and supports C++ classes with non-trivial constructors and destructors. (9001553)

ACKs for top commit:
  jonasschnelli:
    Tested ACK d76894987d0277e8011932ab7dfd77c537f8ea6e
  nijynot:
    ACK d768949
  hebasto:
    ACK d76894987d0277e8011932ab7dfd77c537f8ea6e

Tree-SHA512: 48f3e4104b80bd7b6aedcef10bb1957b073530130f33af7c5cb59e876ac3f5480e53d7af1c0b226d809fe9eef1add3d6c3fb6de4af174966202c6030060ea823
2022-06-18 22:14:01 -07:00
fanquake
262671182d
Merge #17362: test: speed up wallet_avoidreuse, add logging
0e7c90eb37a687158c261ddd1ff9f1028a1e7012 test: speed up wallet_avoidreuse.py (Jon Atack)
6d50b2606ea9249627556051637080c3587b1b04 test: add logging to wallet_avoidreuse.py (Jon Atack)

Pull request description:

  Inspired by PRs #17340 and #15881.

  - add logging
  - pass -whitelist in `set_test_params` to speed up transaction relay

  `wallet_avoidreuse.py` is not intended to test P2P transaction relay/timing, so it should be fine to do this here. This reduces test run time variability and speeds up the test by 2-3 times on average.

  Test run times in seconds:

  - before: 20, 24, 22, 17, 27, 40, 30

  - after: 10, 10, 8, 9, 10, 7, 8

ACKs for top commit:
  MarcoFalke:
    ACK 0e7c90eb37a687158c261ddd1ff9f1028a1e7012 🐊
  fanquake:
    ACK 0e7c90eb37a687158c261ddd1ff9f1028a1e7012

Tree-SHA512: 6d954a0aaf402c9594201626b59d29263479059e68fa5155bb44ed973cd0c3347729dd78b78b4d5a2275e45da365dc1afb4cc7e3293dea33fcc2e3e83a39faf5
2022-06-18 22:14:01 -07:00
PastaPastaPasta
951398e8b3
Merge pull request #4792 from vijaydasmp/bp2007
Merge #16949,#15084,#17138,#16689,#15098, #15932
2022-06-18 22:12:07 -07:00
PastaPastaPasta
aa97e52f26
Merge pull request #4887 from knst/bc-bp-4
Bitcoin backports #13756 #15770 #17730 #17750 #17752 #17826 #17857 #15886
2022-06-18 20:23:31 -07:00
Kittywhiskers Van Gogh
282b02e6b2
rpc: split spork manipulation logic to distinct "sporkupdate" call (#4885)
* rpc: split spork manipulation logic to distinct "sporkupdate" call

Co-authored-by: UdjinM6 <UdjinM6@users.noreply.github.com>

* docs: add release notes for dash#4885

Co-authored-by: UdjinM6 <UdjinM6@users.noreply.github.com>
2022-06-18 19:52:45 +03:00
fanquake
746ff12632
Merge #16949: build: only pass --disable-dependency-tracking to packages that understand it
1ba49bcdc216162f513e74e371ee9f26418a390e build: pass --enable-option-checking to applicable packages (fanquake)
bcff8e21b1bb360c3cf1e1d2e1a2a54ec7581720 build: only pass --disable-dependency-tracking to packages that understand it (fanquake)

Pull request description:

  By blanket passing `--disable-dependency-tracking` to all depends packages we end up with warnings (i.e in `bdb` or `freetype`) like:
  ```bash
  configure: WARNING: unrecognized options: --disable-dependency-tracking
  ```
  Instead, only pass it to packages that actually understand it. Related to https://github.com/bitcoin/bitcoin/issues/16354.

  More info on `--disable-dependency-tracking` available [here](https://www.gnu.org/software/automake/manual/html_node/Dependency-Tracking.html).

  This PR also adds `--enable-option-checking` as a configure option to all applicable packages.

ACKs for top commit:
  laanwj:
    ACK 1ba49bcdc216162f513e74e371ee9f26418a390e
  theuni:
    ACK 1ba49bcdc216162f513e74e371ee9f26418a390e

Tree-SHA512: 6d3143ad5f5d1abed5e0a0b2ffbb4323f21c7bf24b0b8df26fb1b3cd16cf5309bbb830aa5aaec99164d5bbe8e9c62b97aa3e97ee1ddc2c7612bf8ff88a63885e
2022-06-18 17:19:48 +05:30
Jonas Schnelli
59f9ea5815
Merge #15084: gui: don't disable the sync overlay when wallet is disabled
b3b6b6f62fcabea9818e8049dba714d0d0ef8ab6 gui: don't disable the sync overlay when wallet is disabled (Ben Carman)

Pull request description:

  Continuation of #13848.

  When running with `-disablewallet` the sync modal is now available by clicking on the progress bar or `syncing` icon.

  [Current Image of what the window looks like](https://imgur.com/6LsoT2l)

  Fixes #13828.

ACKs for top commit:
  jonasschnelli:
    Tested ACK b3b6b6f62fcabea9818e8049dba714d0d0ef8ab6

Tree-SHA512: 325bc22a0b692bfb8fcc9d84e02dfc506146028b97b3609e23c2c45288c79b8aead1ad2e9b8d692f5f6771b4d2aee63fbe71bfaeaf17d260865da32ab3631e07
2022-06-18 17:19:47 +05:30
MarcoFalke
8853b6e1e4
(partial) Merge #17138: Remove wallet access to some node arguments
b96ed0396294fc4fa89d83ceab6bc169dd09f002 [wallet] Remove pruning check for -rescan option (John Newbery)
eea462de9c652dca556ad241d2126b10790f67f8 [wallet] Remove package limit config access from wallet (John Newbery)

Pull request description:

  Removes wallet access to `-limitancestorcount`, `-limitdescendantcount` and `-prune`:

  - `-limitancestorcount` and `-limitdescendantcount` are now accessed with a method `getPackageLimits` in the `Chain` interface.
  - `-prune` is not required. It was only used in wallet component initiation to prevent running `-rescan` when pruning was enabled. This check is not required.

  Partially addresses #17137.

ACKs for top commit:
  MarcoFalke:
    Tested ACK b96ed0396294fc4fa89d83ceab6bc169dd09f002
  ryanofsky:
    Code review ACK b96ed0396294fc4fa89d83ceab6bc169dd09f002
  promag:
    Code review ACK b96ed0396294fc4fa89d83ceab6bc169dd09f002.
  ariard:
    ACK b96ed03, check there isn't left anymore wallet access to node arguments.

Tree-SHA512: 90c8e3e083acbd37724f1bccf63dab642cf9ae95cc5e684872a67443ae048b4fdbf57b52ea47c5a1da6489fd277278fe2d9bbe95e17f3d4965a1a0fbdeb815bf
2022-06-18 17:19:47 +05:30
MarcoFalke
695869f0b9
Merge #16689: rpc: add missing fields to wallet rpc help output
f4c8953b0073e6bc37abd18ec4a5cbc3ea9719a1 Add missing fields in TransactionDescriptionString and others (Antoine Riard)
353010849185cda53c97da1f5089565dccf6f16a MOVEONLY : move RPC wallets helpers to TransactionDescriptionString (Antoine Riard)

Pull request description:

  Knowledge of `walletconflicts` field existence is really nice when you're debugging conflicts. Was added in #3671 but never documented in RPC helps.

  Others were added after a quick skim, we may still have missing ones in wallet rpcs.

ACKs for top commit:
  MarcoFalke:
    re-ACK f4c8953b0073e6bc37abd18ec4a5cbc3ea9719a1 (only change is addressing my nits)
  kristapsk:
    ACK f4c8953b0073e6bc37abd18ec4a5cbc3ea9719a1

Tree-SHA512: 2bea4d8743399fb152f942df7454548b896b2ad5654fd4bf60253afec1a5387ef8797ced97776dc7ba4912291263c08abe7c2b608c6a28f9a0df67be4ebc4635
2022-06-18 17:19:47 +05:30
fanquake
933c629113
Merge #15098: qt: Show addresses for "SendToSelf" transactions
11fdfcf7f940fab48625d102e825a59c16ad4fbc Show addresses for "SendToSelf" transactions (Hennadii Stepanov)

Pull request description:

  Fix #11464
  Fix #12688
  Ref: #11471 by jonasschnelli

  Note: change addresses are not recognized (ref: https://github.com/bitcoin/bitcoin/pull/11471#discussion_r180547041)

  Result:
  ![screenshot from 2019-01-04 09-06-12](https://user-images.githubusercontent.com/32963518/50677356-19c79500-1002-11e9-986c-999c366e4b2a.png)

ACKs for top commit:
  jonasschnelli:
    Tested ACK 11fdfcf7f940fab48625d102e825a59c16ad4fbc
  fanquake:
    ACK 11fdfcf7f940fab48625d102e825a59c16ad4fbc - did the bare minimum testing.

Tree-SHA512: 2678a2fdf017c376750c73fdc751b7838b0d3a970ba02e9032e4c5824494362672036c3ebf87b425aefdfe197fb952b70e4b7b6011077abb39a8bfc1ae14dfd2
2022-06-18 17:19:46 +05:30
MarcoFalke
d5986a1e8f
Merge #15932: rpc: Serialize in getblock without cs_main
faea56400d rpc: Add lock annotations to block{,header}ToJSON (MarcoFalke)
fab00a5cb9 rpc: Serialize in getblock without cs_main (MarcoFalke)
fa1c3591ad rpc: Use IsValidNumArgs in getblock (MarcoFalke)

Pull request description:

  No need to hold cs_main when serializing a struct to json

  Fixes: #15925

ACKs for commit faea56:
  jnewbery:
    utACK faea56400d5578023133cf4d1c761cdeb0c3e3da
  jonasschnelli:
    utACK faea56400d5578023133cf4d1c761cdeb0c3e3da

Tree-SHA512: 005d378cda1e6024e9f5142f99a8adbefe202cd7bfeaafee55eb909e8990a3790aa27fcf5dd16119cc9afe9dc8bd30f660de40233316781669be166bac3018e7
2022-06-18 11:37:52 +05:30
Wladimir J. van der Laan
77cf41116a Merge #17826: qt: Log Qt related info
a004673c54d4c015775e0baced21f3fa961bf754 qt: Add LogQtInfo() function (Hennadii Stepanov)

Pull request description:

  This PR adds some info to `debug.log` I found useful for testing (e.g., on Wayland) and debugging issues like #17153:

  ```
  $ ./src/qt/bitcoin-qt -printtoconsole | head -n 6
  2020-01-04T14:57:40Z [main] Bitcoin Core version v0.19.99.0-0df287f4e (release build)
  2020-01-04T14:57:40Z [main] InitParameterInteraction: parameter interaction: -externalip set -> setting -discover=0
  2020-01-04T14:57:40Z [main] Qt 5.9.5 (dynamic), plugin=xcb (dynamic)
  2020-01-04T14:57:40Z [main] System: Linux Mint 19.3, x86_64-little_endian-lp64
  2020-01-04T14:57:40Z [main] Screen: HDMI-1 1600x1200, pixel ratio=1.0
  2020-01-04T14:57:40Z [main] Assuming ancestors of block 00000000000000b7ab6ce61eb6d571003fbe5fe892da4c9b740c49a07542462d have valid signatures.
  ```

ACKs for top commit:
  laanwj:
    ACK a004673c54d4c015775e0baced21f3fa961bf754

Tree-SHA512: 496bcfd4870a2730eab92b96b3e74989a7904b21369c372b6d4368f4ca2c141e2fdc1348a1fdd18cb68bb144dcea01d3023bb782f9d030e330c187f6a5a1a082
2022-06-17 04:06:49 +07:00
fanquake
7b0ff6ce10 Merge #17857: scripts: fix symbol-check & security-check argument passing
71af793512100ee7d508c3fb815af47925fe80ba scripts: fix check-symbols & check-security argument passing (fanquake)

Pull request description:

  The first argument in `bin_PROGRAMS` (`bitcoind`) was being silently consumed and never passed into the [`security-check.py`](https://github.com/bitcoin/bitcoin/blob/master/contrib/devtools/security-check.py) or [`symbol-check.py`](https://github.com/bitcoin/bitcoin/blob/master/contrib/devtools/symbol-check.py) scripts.

  This seems to have been the case since the scripts were added to the makefile in f3d3eaf78e.

  Example of the behavior:

  ```python
  # touch a, touch b, touch c
  # python3 args.py < a b c

  import sys
  if __name__ == '__main__':
      print(sys.argv)
      # ['args.py', 'b', 'c']

      # if you add some lines to "a",
      # you'll see them here..
      for line in sys.stdin:
          print(line)
  ```

ACKs for top commit:
  laanwj:
    ACK 71af793512100ee7d508c3fb815af47925fe80ba

Tree-SHA512: 9d0b975a11f66fd87a76654d210808000a629c9cce4c760f71e8a2bcb4e99b9109419f2306db67cf9b12c28e40b96ae722b7c9b4569b2b8bacd469fb99db30c3
2022-06-17 04:06:49 +07:00
MeshCollider
6099b4bf39 Merge #13756: wallet: "avoid_reuse" wallet flag for improved privacy
5ebc6b0eb267e0552c66fffc5e5afe7df8becf80 bitcoind: update -avoidpartialspends description to account for auto-enable for avoid_reuse wallets (Karl-Johan Alm)
ada258f8c8f92d44d893cf9f22d15acdeca40b1a doc: release notes for avoid_reuse (Karl-Johan Alm)
27669551da52099e4a6a401acd7aa32b32832423 wallet: enable avoid_partial_spends by default if avoid_reuse is set (Karl-Johan Alm)
8f2e208f7c0468f9ba92bc789a698281b1c81284 test: add test for avoidreuse feature (Karl-Johan Alm)
0bdfbd34cf4015de87741ff549db35e5064f4e16 wallet/rpc: add 'avoid_reuse' option to RPC commands (Karl-Johan Alm)
f904723e0d5883309cb0dd14b826bc45c5e776fb wallet/rpc: add setwalletflag RPC and MUTABLE_WALLET_FLAGS (Karl-Johan Alm)
8247a0da3a46d7c38943ee0304343ab7465305bd wallet: enable avoid_reuse feature (Karl-Johan Alm)
eec15662fad917b169f5e3b8baaf4301dcf00a7b wallet: avoid reuse flags (Karl-Johan Alm)
58928098c299efdc7c5ddf2dc20716ca5272f21b wallet: make IsWalletFlagSet() const (Karl-Johan Alm)
129a5bafd9a3efa2fa16d780885048a06566d262 wallet: rename g_known_wallet_flags constant to KNOWN_WALLET_FLAGS (Karl-Johan Alm)

Pull request description:

  Add a new wallet flag called `avoid_reuse` which, when enabled, will keep track of when a specific destination has been spent from, and will actively "blacklist" any new UTXOs which send to an already-spent-from destination.

  This improves privacy, as a payer could otherwise begin tracking a payee's wallet by regularly peppering a known UTXO with dust outputs, which would then be scooped up and used in payments by the payee, allowing the payer to map out (1) the inputs owned by the payee and (2) the destinations to which the payee is making payments.

  This replaces #10386 and together with the (now merged) #12257 it addresses #10065 in full. The concerns raised in https://github.com/bitcoin/bitcoin/pull/10386#issuecomment-302361381 are also addressed due to #12257.

  ~~Note: this builds on top of #15780.~~ (merged)

ACKs for commit 5ebc6b:
  jnewbery:
    ACK 5ebc6b0eb
  laanwj:
    Concept and code-review ACK 5ebc6b0eb267e0552c66fffc5e5afe7df8becf80
  meshcollider:
    Code review ACK 5ebc6b0eb2
  achow101:
    ACK 5ebc6b0eb267e0552c66fffc5e5afe7df8becf80 modulo above nits

Tree-SHA512: fdef45826af544cbbb45634ac367852cc467ec87081d86d08b53ca849e588617e9a0a255b7e7bb28692d15332de58d6c3d274ac003355220e4213d7d9070742e
2022-06-17 04:06:49 +07:00
Konstantin Akimov
6e5440d505 Removed some extra qt dependencies (seems missing in previous backports) 2022-06-17 04:06:49 +07:00
Wladimir J. van der Laan
a91a406890 Merge #17730: depends: remove Qt networking features
244501fc85a1319857efb227093c0e71c1d5a01e depends: disable unused qt networking features (fanquake)
29d56c62b7f206d42f0908819ff2e1926737f988 depends: -optimized-qmake is now -optimized-tools (fanquake)
ccdda96804088ec3ad01aec5ab0ff8e9b05b161b depends: skip building qt proxies (fanquake)

Pull request description:

  Somewhat of a followup to removing BIP70 support in #17165. This removes networking features from our Qt build. This also removes the need to link against the `CFNetwork` and `SystemConfiguration` libraries on macOS.

  ```diff
  src/qt/bitcoin-qt:
   /usr/lib/libSystem.B.dylib
   /System/Library/Frameworks/IOKit.framework/Versions/A/IOKit
   /System/Library/Frameworks/Foundation.framework/Versions/C/Foundation
   /System/Library/Frameworks/CoreServices.framework/Versions/A/CoreServices
   /System/Library/Frameworks/AppKit.framework/Versions/C/AppKit
   /System/Library/Frameworks/ApplicationServices.framework/Versions/A/ApplicationServices
   /System/Library/Frameworks/CoreFoundation.framework/Versions/A/CoreFoundation
  -/System/Library/Frameworks/SystemConfiguration.framework/Versions/A/SystemConfiguration
   /System/Library/Frameworks/CoreGraphics.framework/Versions/A/CoreGraphics
   /System/Library/Frameworks/Carbon.framework/Versions/A/Carbon
   /usr/lib/libc++.1.dylib
  -/System/Library/Frameworks/CFNetwork.framework/Versions/A/CFNetwork
   /System/Library/Frameworks/CoreText.framework/Versions/A/CoreText
   /System/Library/Frameworks/ImageIO.framework/Versions/A/ImageIO
   /usr/lib/libobjc.A.dylib
  ```

  > Introduced the -optimized-tools option; supersedes -optimized-qmake.

  `optimized-qmake` became `optimized-tools` in Qt 5.6.0. While the former still works, we can use the newer flag.

  A diff of the removed symbols is available [here](https://gist.github.com/fanquake/9c8d5961c91f90a2966191367adfb391).

  We still need to actually build the network module, because we are using `QLocalServer` & `QLocalSocket` in the payment server.

ACKs for top commit:
  Sjors:
    Code review ACK 244501fc85a1319857efb227093c0e71c1d5a01e: just a rebase (_updated since I accidentally repeated the previous hash_)
  practicalswift:
    ACK 244501fc85a1319857efb227093c0e71c1d5a01e -- diff looks correct
  promag:
    Code review ACK 244501fc85a1319857efb227093c0e71c1d5a01e.

Tree-SHA512: 79734e3c96c40e7e484c86ac4cd4f738c05fcebe4771aeac443883f618a6c766e667909d5f8f14f9bd82f43206387c952458c5fa765cd0830f8beda6e6ac80ae
2022-06-17 04:06:49 +07:00
Wladimir J. van der Laan
a081afd3c1 Merge #15886: qt, wallet: Revamp SendConfirmationDialog
78f9b5160fea32f85c6ced16b3b0cccda6d3b181 Do not show list for the only recipient. (Hennadii Stepanov)
2ee756f0411c467fc638373ea68d6cb5ced46974 Show recipient list as detailedText of QMessageBox (Hennadii Stepanov)
654e419549fb5f1c360843d48e66eb098bbf5a83 Make SendConfirmationDialog fully fledged (Hennadii Stepanov)

Pull request description:

  Fix #15667

  With this PR:
  ![Screenshot from 2019-04-24 23-47-30](https://user-images.githubusercontent.com/32963518/56692672-63400b00-66eb-11e9-87f6-15957c6e81f7.png)
  ![Screenshot from 2019-04-24 23-47-40](https://user-images.githubusercontent.com/32963518/56692681-663afb80-66eb-11e9-8b04-8a342026ada6.png)

ACKs for commit 78f9b5:
  laanwj:
    code review ACK 78f9b5160fea32f85c6ced16b3b0cccda6d3b181

Tree-SHA512: f868d78d01b0898aff2277fa3a7e8c6f936acbbcfa8a0323cddcd9daba4a998030c667bd803ae67c2b9179ed8082a48a67568e9ba3c8d14e3a2d88d93ada94fa
2022-06-17 04:06:49 +07:00
PastaPastaPasta
e090d9a9e7
trivial(llmq): trivial fixes in llmq/utils.cpp (#4884)
* trivial(llmq): fix indentation

* trivial(llmq): remove unused / commented out includes

* trivial(llmq): adjust formatting

* trivial(llmq): use const

* trivial(llmq): use ranges

* trivial: fix linter
2022-06-16 00:00:43 +03:00
MarcoFalke
6e709d48c4 Merge #15770: rpc: Validate maxfeerate with AmountFromValue
aa410c2b17 rpc: Validate maxfeerate with AmountFromValue (João Barbosa)

Pull request description:

  With this change `maxfeerate` can also be set as a string, accordingly to the help test:
  ```
    maxfeerate    (numeric or string,
  ```
  Beside, there are no tests for the removed errors.

ACKs for commit aa410c:
  meshcollider:
    utACK aa410c2b17
  MarcoFalke:
    utACK aa410c2b17 Good catch

Tree-SHA512: f3bfea91dc7daa943729e270585dbf333055aeda805fbd01eaab20a7e0e6147382647c11525334382d198df0d3d45da6102b541efda5a1361f96271c98d5d89d
2022-06-14 16:48:23 +07:00
MarcoFalke
376a6a04f1 Merge #17750: util: change GetWarnings parameter to bool
7aab8d1024996c7c422bd34a8226df0117b813f7 [style] Code style fixups in GetWarnings() (John Newbery)
492c6dc1e742a62599dc6d5ba6c3896825b5144f util: change GetWarnings parameter to bool (John Newbery)
869b6314fd180856b6054fff28b5de994252c54c [qt] remove unused parameter from getWarnings() (John Newbery)

Pull request description:

  `GetWarnings()` changes the format of the output warning string based on a passed-in string argument that can be set to "gui" or "statusbar".

  Change the argument to a bool:

  - there are only two types of behaviour, so a bool is a more natural argument type
  - changing the name to `verbose` does not set any expectations for the how the calling code will use the returned string (currently, `statusbar` is used for RPC warnings, not a status bar)
  - removes some error-handling code for when the passed-in string is not one of the two strings expected.

ACKs for top commit:
  laanwj:
    code review ACK 7aab8d1024996c7c422bd34a8226df0117b813f7
  practicalswift:
    ACK 7aab8d1024996c7c422bd34a8226df0117b813f7 -- diff looks correct :)
  MarcoFalke:
    ACK 7aab8d1024996c7c422bd34a8226df0117b813f7 otherwise.
  promag:
    Code review ACK 7aab8d1024996c7c422bd34a8226df0117b813f7.

Tree-SHA512: 75882c6e3e44aa9586411b803149b36ba487f4eb9cac3f5c8f07cd9f586870bba4488a51e674cf8147f05718534f482836e6a4e3f66e0d4ef6821900c7dfd04e
2022-06-14 16:48:15 +07:00
fanquake
f286654862 Merge #17752: doc: fix directory path for secp256k1 subtree in developer-notes
a5089f62bda9a39c1d6cbba285477670f1aa1f3f fix directory path for secp256k1 subtree in developer-notes (hackerrdave)

Pull request description:

  Documentation update to fix the directory path of the `secp256k1` subtree in the developer notes

ACKs for top commit:
  laanwj:
    ACK a5089f62bda9a39c1d6cbba285477670f1aa1f3f

Tree-SHA512: d0986721d7091af26edaee769db78c9aabac25bbaddb2a1bfa96c7208187226e280e9c38897b5227ee6c9e40d5a1af86bb7c58e72c6a30a94a478c4bf54c086e
2022-06-14 16:48:08 +07:00
Nathan Marley
d43f9d4ae1
fix(gov): do not allow empty proposal names (#4883)
* gov: Do not allow empty proposal names

* Add test for invalid (empty) proposal name

* Use pasta suggestion
2022-06-14 12:04:56 +03:00
UdjinM6
bcc4b2d1ac
Merge pull request #4867 from knst/bc-bp-3
Bitcoin backports #12763 #17511 #17675 #17617 #17698 #17703 #17705 #17787 #17806 #17743
2022-06-14 12:02:25 +03:00
Wladimir J. van der Laan
0f13a30378 Merge #17743: doc: Add release note for RPC Whitelist
7965e0b41ae03110ad784a1b374f831600c0cca1 doc: Add release note for RPC Whitelist (Emil Engler)

Pull request description:

  A release note for #12763

ACKs for top commit:
  laanwj:
    ACK 7965e0b41ae03110ad784a1b374f831600c0cca1

Tree-SHA512: 4ac3e62029a403e64e4cd3183433dc7aa071d42688b689d7cffb8f08dc4b26d2a586d32fa791d2b5679d6b95cd6e34c56e40a5592b9af446ad9429307f7267fe
2022-06-13 14:35:18 +07:00
UdjinM6
9958ba6ea0
refactor(qt): Reuse QRImageWidget in QRDialog (#4853)
4570 follow-up (15928 backport)
2022-06-11 12:18:01 +03:00
Nathan Marley
fc794738af
trivial: Fix typo in gobject getcurrentvotes help (#4881)
This help text didn't make sense to me 'til I realized it should be `present`. I kept wondering what txid should be presenting...
2022-06-11 12:16:15 +03:00
Odysseas Gabrielides
0db7c7d709
docs(llmq): Improved documentation for keepOldConnections (#4876) 2022-06-10 23:26:02 -05:00
Munkybooty
7097171123
tests: basic functional tests for CoinJoin (#4730)
* refactor: create separate function for processing dsq messages in client.cpp

* refactor: avoid duplicate checks in CCoinJoinClientSession::ProcessMessage

* test: create tests for mixing and pending dsa requests in coinjoin_tests.cpp

* test: create tests for CTxDSIn in coinjoin_tests.cpp

* test: create tests for CoinJoinAccept in coinjoin_tests.cpp

* test: create tests for CoinJoinStatusUpdate in coinjoin_tests.cpp

* removal of check peer version

* Apply suggestions from code review

Co-authored-by: UdjinM6 <UdjinM6@users.noreply.github.com>
2022-06-10 23:05:48 -05:00
UdjinM6
7fb8ae7d27
Merge pull request #4874 from UdjinM6/addrv2_followups
backport: bitcoin 20564, 20661 (addrv2 follow-ups)
2022-06-08 13:01:44 +03:00
Konstantin Akimov
25f78025f5 Fixed base58 fuzz tests, part of #17511 2022-06-08 12:36:52 +07:00
Konstantin Akimov
56a9f9b0dd Fixed fuzz build on Kubuntu 22.04 2022-06-08 12:36:52 +07:00