Commit Graph

1064 Commits

Author SHA1 Message Date
fanquake
bd7b5a3ad4
Merge #18713: scripts: Add MACHO stack canary check to security-check.py
8334ee31f868f0f9baf0920d14d20174ed889dbe scripts: add MACHO LAZY_BINDINGS test to test-security-check.py (fanquake)
7b99c7454cdb74cd9cd7a5eedc2fb9d0a19df456 scripts: add MACHO Canary check to security-check.py (fanquake)

Pull request description:

  7b99c7454cdb74cd9cd7a5eedc2fb9d0a19df456 uses `otool -Iv` to check for `___stack_chk_fail` in the macOS binaries. Similar to the [ELF check](https://github.com/bitcoin/bitcoin/blob/master/contrib/devtools/security-check.py#L105). Note that looking for a triple underscore prefixed function (as opposed to two for ELF) is correct for the macOS binaries. i.e:
  ```bash
  otool -Iv bitcoind | grep chk
  0x00000001006715b8   509 ___memcpy_chk
  0x00000001006715be   510 ___snprintf_chk
  0x00000001006715c4   511 ___sprintf_chk
  0x00000001006715ca   512 ___stack_chk_fail
  0x00000001006715d6   517 ___vsnprintf_chk
  0x0000000100787898   513 ___stack_chk_guard
  ```

  8334ee31f868f0f9baf0920d14d20174ed889dbe is a follow up to #18295 and adds test cases to `test-security-check.py` that for some reason I didn't add at the time. I'll sort out #18434 so that we can run these tests in the CI.

ACKs for top commit:
  practicalswift:
    ACK 8334ee31f868f0f9baf0920d14d20174ed889dbe: Mitigations are important. Important things are worth asserting :)
  jonasschnelli:
    utACK 8334ee31f868f0f9baf0920d14d20174ed889dbe.

Tree-SHA512: 1aa5ded34bbd187eddb112b27278deb328bfc21ac82316b20fab6ad894f223b239a76b53dab0ac1770d194c1760fcc40d4da91ec09959ba4fc8eadedb173936a
2022-06-18 22:14:03 -07:00
fanquake
67ceda1b5a
Merge #18295: scripts: add MACHO lazy bindings check to security-check.py
5ca90f8b598978437340bb8467f527b9edfb2bbf scripts: add MACHO lazy bindings check to security-check.py (fanquake)

Pull request description:

  This is a slightly belated follow up to #17686 and some discussion with Cory. It's not entirely clear if we should make this change due to the way the macOS dynamic loader appears to work. However I'm opening this for some discussion. Also related to #17768.

  #### Issue:
  [`LD64`](https://opensource.apple.com/source/ld64/) doesn't set the [MH_BINDATLOAD](https://opensource.apple.com/source/xnu/xnu-6153.11.26/EXTERNAL_HEADERS/mach-o/loader.h.auto.html) bit in the header of MACHO executables, when building with `-bind_at_load`. This is in contradiction to the [documentation](https://opensource.apple.com/source/ld64/ld64-450.3/doc/man/man1/ld.1.auto.html):
  ```bash
  -bind_at_load
       Sets a bit in the mach header of the resulting binary which tells dyld to
       bind all symbols when the binary is loaded, rather than lazily.
  ```

  The [`ld` in Apples cctools](https://opensource.apple.com/source/cctools/cctools-927.0.2/ld/layout.c.auto.html) does set the bit, however the [cctools-port](https://github.com/tpoechtrager/cctools-port/) that we use for release builds, bundles `LD64`.

  However; even if the linker hasn't set that bit, the dynamic loader ([`dyld`](https://opensource.apple.com/source/dyld/)) doesn't seem to ever check for it, and from what I understand, it looks at a different part of the header when determining whether to lazily load symbols.

  Note that our release binaries are currently working as expected, and no lazy loading occurs.

  #### Example:

  Using a small program, we can observe the behaviour of the dynamic loader.

  Conducted using:
  ```bash
  clang++ --version
  Apple clang version 11.0.0 (clang-1100.0.33.17)
  Target: x86_64-apple-darwin18.7.0

  ld -v
  @(#)PROGRAM:ld  PROJECT:ld64-530
  BUILD 18:57:17 Dec 13 2019
  LTO support using: LLVM version 11.0.0, (clang-1100.0.33.17) (static support for 23, runtime is 23)
  TAPI support using: Apple TAPI version 11.0.0 (tapi-1100.0.11)
  ```

  ```cpp
  #include <iostream>
  int main() {
  	std::cout << "Hello World!\n";
  	return 0;
  }
  ```

  Compile and check the MACHO header:
  ```bash
  clang++ test.cpp -o test
  otool -vh test
  ...
  Mach header
        magic cputype cpusubtype  caps    filetype ncmds sizeofcmds      flags
  MH_MAGIC_64  X86_64        ALL LIB64     EXECUTE    16       1424   NOUNDEFS DYLDLINK TWOLEVEL WEAK_DEFINES BINDS_TO_WEAK PIE

  # Run and dump dynamic loader bindings:
  DYLD_PRINT_BINDINGS=1 DYLD_PRINT_TO_FILE=no_bind.txt ./test
  Hello World!
  ```

  Recompile with `-bind_at_load`. Note still no `BINDATLOAD` flag:
  ```bash
  clang++ test.cpp -o test -Wl,-bind_at_load
  otool -vh test
  Mach header
        magic cputype cpusubtype  caps    filetype ncmds sizeofcmds      flags
  MH_MAGIC_64  X86_64        ALL LIB64     EXECUTE    16       1424   NOUNDEFS DYLDLINK TWOLEVEL WEAK_DEFINES BINDS_TO_WEAK PIE
  ...
  DYLD_PRINT_BINDINGS=1 DYLD_PRINT_TO_FILE=bind.txt ./test
  Hello World!
  ```

  If we diff the outputs, you can see that `dyld` doesn't perform any lazy bindings when the binary is compiled with `-bind_at_load`, even if the `BINDATLOAD` flag is not set:
  ```diff
  @@ -1,11 +1,27 @@
  +dyld: bind: test:0x103EDF030 = libc++.1.dylib:__ZNKSt3__16locale9use_facetERNS0_2idE, *0x103EDF030 = 0x7FFF70C9FA58
  +dyld: bind: test:0x103EDF038 = libc++.1.dylib:__ZNKSt3__18ios_base6getlocEv, *0x103EDF038 = 0x7FFF70CA12C2
  +dyld: bind: test:0x103EDF068 = libc++.1.dylib:__ZNSt3__113basic_ostreamIcNS_11char_traitsIcEEE6sentryC1ERS3_, *0x103EDF068 = 0x7FFF70CA12B6
  +dyld: bind: test:0x103EDF070 = libc++.1.dylib:__ZNSt3__113basic_ostreamIcNS_11char_traitsIcEEE6sentryD1Ev, *0x103EDF070 = 0x7FFF70CA1528
  +dyld: bind: test:0x103EDF080 = libc++.1.dylib:__ZNSt3__16localeD1Ev, *0x103EDF080 = 0x7FFF70C9FAE6
  <trim>
  -dyld: lazy bind: test:0x10D4AC0C8 = libsystem_platform.dylib:_strlen, *0x10D4AC0C8 = 0x7FFF73C5C6E0
  -dyld: lazy bind: test:0x10D4AC068 = libc++.1.dylib:__ZNSt3__113basic_ostreamIcNS_11char_traitsIcEEE6sentryC1ERS3_, *0x10D4AC068 = 0x7FFF70CA12B6
  -dyld: lazy bind: test:0x10D4AC038 = libc++.1.dylib:__ZNKSt3__18ios_base6getlocEv, *0x10D4AC038 = 0x7FFF70CA12C2
  -dyld: lazy bind: test:0x10D4AC030 = libc++.1.dylib:__ZNKSt3__16locale9use_facetERNS0_2idE, *0x10D4AC030 = 0x7FFF70C9FA58
  -dyld: lazy bind: test:0x10D4AC080 = libc++.1.dylib:__ZNSt3__16localeD1Ev, *0x10D4AC080 = 0x7FFF70C9FAE6
  -dyld: lazy bind: test:0x10D4AC070 = libc++.1.dylib:__ZNSt3__113basic_ostreamIcNS_11char_traitsIcEEE6sentryD1Ev, *0x10D4AC070 = 0x7FFF70CA1528
  ```

  Note: `dyld` also has a `DYLD_BIND_AT_LAUNCH=1` environment variable, that when set, will force any lazy bindings to be non-lazy:
  ```bash
  dyld: forced lazy bind: test:0x10BEC8068 = libc++.1.dylib:__ZNSt3__113basic_ostream
  ```

  #### Thoughts:
  After looking at the dyld source, I can't find any checks for `MH_BINDATLOAD`. You can see the flags it does check for, such as MH_PIE or MH_BIND_TO_WEAK [here](https://opensource.apple.com/source/dyld/dyld-732.8/src/ImageLoaderMachO.cpp.auto.html).

  It seems that the lazy binding of any symbols depends on whether or not [lazy_bind_size](https://opensource.apple.com/source/xnu/xnu-6153.11.26/EXTERNAL_HEADERS/mach-o/loader.h.auto.html) from the `LC_DYLD_INFO_ONLY` load command is > 0. Which was mentioned in [#17686](https://github.com/bitcoin/bitcoin/pull/17686#issue-350216254).

  #### Changes:
  This PR is one of [Corys commits](7b6ba26178), that I've rebased and modified to make build. I've also included an addition to the `security-check.py` script to check for the flag.

  However, given the above, I'm not entirely sure this patch is the correct approach. If the linker no-longer inserts it, and the dynamic loader doesn't look for it, there might be little benefit to setting it. Or, maybe this is an oversight from Apple and needs some upstream discussion. Looking for some thoughts / Concept ACK/NACK.

  One alternate approach we could take is to drop the patch and modify security-check.py to look for `lazy_bind_size` == 0 in the `LC_DYLD_INFO_ONLY` load command, using `otool -l`.

ACKs for top commit:
  theuni:
    ACK 5ca90f8b598978437340bb8467f527b9edfb2bbf

Tree-SHA512: 444022ea9d19ed74dd06dc2ab3857a9c23fbc2f6475364e8552d761b712d684b3a7114d144f20de42328d1a99403b48667ba96885121392affb2e05b834b6e1c
2022-06-18 22:14:02 -07:00
Wladimir J. van der Laan
3c4982205c
Merge #18415: scripts: add MACHO tests to test-security-check.py
7142d50ac33e0ad7d24e49e04c1fc7e3e769ed46 scripts: rename test_64bit_PE to test_PE (fanquake)
edaca2dd123cef958699c07ab248cf0ffc71af07 scripts: add MACHO NX check to security-check.py (fanquake)
1a4e9f32efcc5f6a74290446dc58784fd85c7b31 scripts: add MACHO tests to test-security-check.py (fanquake)

Pull request description:

  Adds tests for the MACHO checks in security-check.py:
  ac579ada7e/contrib/devtools/security-check.py (L212-L214)

  I'm planning on following up with more checks in security-check.py, and corresponding tests in test-security-check.py.

  Note that you'll probably have to be on macOS to run them. You can run just this suite with `python3 test-security-check.py TestSecurityChecks.test_MACHO`.

ACKs for top commit:
  laanwj:
    ACK 7142d50ac33e0ad7d24e49e04c1fc7e3e769ed46

Tree-SHA512: ace3ca9f6df5d4fedd5988938fb7dc7563ec7dc587aa275f780b5f51e9b8d7d6f7768e0a1e05ce438510a07b8640aba92c76847b30c2990f46c66b78a0acf960
2022-06-18 22:14:02 -07:00
Wladimir J. van der Laan
a25ee06cc1 Merge #17787: scripts: add MACHO PIE check to security-check.py
7c9e821c4e6cb186208ead9c8df616d1f393a49a scripts: add MACHO NOUNDEFS check to security-check.py (fanquake)
4ca92dc6d3f3e487d63286d8871d1829b3d279ff scripts: add MACHO PIE check to security-check.py (fanquake)

Pull request description:

  This uses `otool -vh` to print the mach header and look for the `PIE` flag:
  ```bash
  otool -vh src/bitcoind
  Mach header
        magic cputype cpusubtype  caps    filetype ncmds sizeofcmds      flags
  MH_MAGIC_64  X86_64        ALL LIB64     EXECUTE    24       2544   NOUNDEFS DYLDLINK TWOLEVEL WEAK_DEFINES BINDS_TO_WEAK PIE
  ```

  From [`mach-o/loader.h`](https://opensource.apple.com/source/cctools/cctools-927.0.2/include/mach-o/loader.h.auto.html):
  ```c
  #define	MH_PIE 0x200000			/* When this bit is set, the OS will
  					   load the main executable at a
  					   random address.  Only used in
  					   MH_EXECUTE filetypes. */
  ```

ACKs for top commit:
  laanwj:
    code review ACK 7c9e821c4e6cb186208ead9c8df616d1f393a49a

Tree-SHA512: 5ba2f60440d0e31c70371a355c91ca4f723d80f7287d04e2098bf5b11892cc74216ff8f1454603c4db9675d4f7983614843b992b8dcfca0309aadf2aa7ab2e4b
2022-06-08 12:36:52 +07:00
PastaPastaPasta
d64b7229cd
chore: bump copyrights (#4873)
* chore: bump copyright in configure.ac

* chore: bump copyright via copyright_header.py

ran command `python3 contrib/devtools/copyright_header.py update .`
2022-06-08 02:36:46 +03:00
PastaPastaPasta
875305a901
Merge pull request #4839 from Munkybooty/backports-0.20-pr2
backport: v0.20 pr2
2022-06-01 13:53:54 -05:00
Wladimir J. van der Laan
37ef9bb465 Merge #17282: contrib: remove accounts from bash completion
dd94cc46e4377b14515be24894fd1cb42e233151 contrib: remove accounts from bash completion (fanquake)

Pull request description:

  Also removes `setgenerate`.

ACKs for top commit:
  jonasschnelli:
    utACK dd94cc46e4377b14515be24894fd1cb42e233151

Tree-SHA512: 218aa06841f4a347bb083c7f408c07fe80df7a30f7ce7a1126ec95971a9b66aff7c3c049d269da2c15f629bbde80e8c303bf6562bd5c1a36a6f6fd61ce9133e2
2022-05-31 12:06:32 -04:00
Kittywhiskers Van Gogh
3ba8431900 merge bitcoin#16899: UTXO snapshot creation (dumptxoutset)
Co-authored-by: UdjinM6 <UdjinM6@users.noreply.github.com>
2022-05-23 10:38:33 +05:30
strophy
ff2f768114
ci: fix docker context (#4816) 2022-04-28 11:35:50 -05:00
MarcoFalke
a3a7a22268
Merge #20223: build: Drop the leading 0 from the version number
8f7b93047581c67f2133cdb8c7845471de66c30f Drop the leading 0 from the version number (Andrew Chow)

Pull request description:

  Removes the leading 0 from the version number. The minor version, which we had been using as the major version, is now the major version. The revision, which we had been using as the minor version, is now the minor version. The revision number is dropped. The build number is promoted to being part of the version number. This also avoids issues where it was accidentally not included in the version number.

  The CLIENT_VERSION remains the same format as previous as previously, as the Major version was 0 so it never actually got included in it.

  The user agent string formatter is updated to follow this new versioning.

  ***

  Honestly I'm just tired of all of the people asking for "1.0" that maybe this'll shut them up. Skip the whole 1.0 thing and go straight to version 22.0!

  Also, this means that the terminology we commonly use lines up with how the variables are named. So major versions are actually bumping the major version number, etc.

ACKs for top commit:
  jnewbery:
    Code review ACK 8f7b930475
  MarcoFalke:
    review ACK 8f7b93047581c67f2133cdb8c7845471de66c30f 🎻

Tree-SHA512: b5c3fae14d4c0a9c0ab3b1db7c949ecc0ac3537646306b13d98dd0efc17c489cdd16d43f0a24aaa28e9c4a92ea360500e05480a335b03f9fb308010cdd93a436
2022-04-28 13:47:53 +03:00
PastaPastaPasta
b421cfacae
perf: enable more multi-threading and caching in linters (#4807)
* perf: enable more multi-threading and caching in linters

20s -> 6s

* ci: add multiprocess to ci dockerfile

* Update test/lint/lint-cppcheck-dash.sh

Co-authored-by: UdjinM6 <UdjinM6@users.noreply.github.com>

Co-authored-by: UdjinM6 <UdjinM6@users.noreply.github.com>
2022-04-27 21:14:40 +03:00
UdjinM6
c6e724bf21 partial bitcoin#22244: Add xkb version to symbol-check 2022-04-26 20:37:20 +05:30
Kittywhiskers Van Gogh
7b69a4cb42 merge bitcoin#21654: Make Qt rcc output always deterministic 2022-04-26 20:36:54 +05:30
Kittywhiskers Van Gogh
b9b9473f55 merge bitcoin#21655: No longer need to set QT_RCC_TEST=1 for determinism 2022-04-26 20:36:54 +05:30
Kittywhiskers Van Gogh
1763f727b5 merge bitcoin#21376: Qt 5.12.10 2022-04-26 00:54:10 +05:30
Kittywhiskers Van Gogh
43152b2b35
merge #17165: Remove BIP70 support (#4023)
* compat: remove bswap_* check on macOS

This was originally added in #9366 to fix the gui build, as
Protobuf would also define these macros. Now that we're no-longer
using Protobuf, remove the additional check.

* build: skip building OpenSSL lib_ssl

* build: remove OpenSSL from Qt build

More info available from:
https://doc.qt.io/qt-5/ssl.html#enabling-and-disabling-ssl-support

* build: remove EVP_MD_CTX_new detection

This was added in #9475 to fix LibreSSL compatibility for
BIP70, so is no longer required.

* build: remove SSL lib detection

* gui: update BIP70 support message

* build: remove BIP70 entries from macOS Info.plist

* gui: remove payment request file handling from OpenURI dialog

* gui: remove BIP70 Support

* build: remove protobuf from depends and contrib
2022-04-25 12:01:47 +03:00
ktechmidas
712f60dd21
Fix: Use github workspace for building image (#4775)
* Fix: Use github workspace for building image

* Fix: Ensure correct dir for docker-entrypoint.sh
2022-04-19 09:11:04 +03:00
UdjinM6
89f606a927
trivial: add some missing dashifications (#4772) 2022-04-19 09:09:42 +03:00
pasta
728699d2cc chore: bump version in gitian-descriptors and README.md 2022-04-16 07:59:23 -06:00
UdjinM6
903291788c
Merge pull request #4735 from kittywhiskers/translation
merge bitcoin#16244...18699: translation backports
2022-04-07 13:18:29 +03:00
UdjinM6
eb42d8eee4
Merge pull request #4749 from PastaPastaPasta/develop-trivial-2022-04-03-pr2
trivial backports 2022 04 03 pr2
2022-04-07 13:17:05 +03:00
Kittywhiskers Van Gogh
0f90ea72d3 merge bitcoin#15450: Create wallet menu option 2022-04-06 13:49:58 +05:30
W. J. van der Laan
6525ca37a7 Merge bitcoin/bitcoin#21709: doc: update reduce-memory.md and bitcoin.conf -maxconnections info
300234ab6698277fb261a87be72f89ef94d3697a doc: update bitcoin.conf maxconnections info (Jon Atack)
926827065ffa56b75e9261f63d49b924d4bced0f doc: update reduce-memory.md peer connections info (Jon Atack)

Pull request description:

  This patch updates the documentation in `doc/reduce-memory.md` and `share/examples/bitcoin.conf` regarding the peer connections limits and `-maxconnections`

ACKs for top commit:
  jarolrod:
    re-ACK 300234ab6698277fb261a87be72f89ef94d3697a
  laanwj:
    ACK 300234ab6698277fb261a87be72f89ef94d3697a
  prayank23:
    ACK 300234ab66

Tree-SHA512: 90f53626124afb50706e6a3b644bc7bb800bb7cf41ae7062c20c17b3d9bdf4a8d73b4cf188faec9113d772596f7e4bc36a4a69481cacb92cc55d5956181d0c31
2022-04-05 23:10:05 -05:00
fanquake
c841e6e66f Merge bitcoin/bitcoin#23598: contrib: Specify wb mode when creating mac sdk
655d52a0cd6b3df738dc9843e25a95b8930b4628 contrib: Specify wb mode when creating mac sdk (João Barbosa)

Pull request description:

  Fix the warning:
  ```
  ./contrib/macdeploy/gen-sdk:84: FutureWarning: GzipFile was opened for writing, but this will change in future Python releases.  Specify the mode argument for opening it for writing.
  ```

ACKs for top commit:
  fanquake:
    Tested ACK 655d52a0cd6b3df738dc9843e25a95b8930b4628 with Python 3.10.

Tree-SHA512: 095cd301f211531a8b8f50e7915fe13c7ab0b278fb23dc50a03625cfae9e2fd7a0d8c315fced4af3011552e3e455ce562b7f717a0ed096c4433ddcf24f22b2c9
2022-04-03 18:48:15 -05:00
PastaPastaPasta
e84bf45cef
Merge pull request #4744 from vijaydasmp/bp2003
Merge #17119,1683,17102,17134,17691,17992,18170,18382,16975,18472, 18486,17633,7833
2022-04-03 17:44:16 -05:00
PastaPastaPasta
279686f9ba
Merge pull request #4742 from vijaydasmp/bp2002
Merge #17154,16569,17351,13716,17084,18391,16821,17140
2022-04-03 17:33:44 -05:00
strophy
1fddc53f53
ci: build on-demand docker images for testing on devnets (#4667)
* ci: build alpha action

* ci: use self-hosted runner

* ci: build faster

* ci: multicore dash build

* chore: copy build, dockerize and push

* chore: dockerize build

* chore: remove unnecessary cachedir and checkout steps

* chore: disable qemu

* chore: remove buildx steps now included in AMI

* chore: move binaries in to place for second stage copy

* fix: runner using incorrect build driver

* chore: debug builder info

* chore: more detailed debug

* chore: install qemu and buildx in actions

* fix: indentation

* fix: remove debug code

* feat: multi-arch cross-compiling dockerfile

* chore: remove unnecessary docker layers

* chore: add debug output

* chore: dump context

* fix: use event inputs tag for docker_meta

* chore: dump context again

* fix: context reference syntax

* feat: attempt to use gitian builder

* chore: ff changes from other branches

* chore: disable macOS build

* Revert "chore: ff changes from other branches"

This reverts commit daece1c505.

* chore: restore GHA changes to fix incorrect merge

* chore: remove duplicate code

* chore: bump buildx version to latest

* chore: try to build without signing

* chore: setup tmate to find build

* chore: run tmate on failure

* fix: invalid path to binaries on copy

* chore: remove unnecessary GPG steps

* chore: replace strophy with dashpay

* fix: gha cache not providing speedup

* Update .github/workflows/release_alpha.yml

Co-authored-by: UdjinM6 <UdjinM6@users.noreply.github.com>
2022-04-03 17:24:06 -05:00
Wladimir J. van der Laan
cca430ba9a
Merge #16569: Increase init file stop timeout
7fb7acfc206b4bf8c296d72b66f3bd4fe342fd87 Set init stop timeout to 10 min (setpill)

Pull request description:

  `bitcoind` can take a long time to flush its db cache to disk upon
  shutdown. Systemd sends a `SIGKILL` after a timeout, causing unclean
  shutdowns and triggering a long "Rolling forward" at the next startup.
  Disabling the timeout should prevent this from happening, and does not
  break systemd's `restart` logic.

  Addresses #13736.

ACKs for top commit:
  instagibbs:
    utACK 7fb7acfc20

Tree-SHA512: 16e0ce5a9ecf0628f8d93d68db3f5a78ab36021d9bede05a90c84f144db2e87e17707a6eb910cb7c018c265ce2c81d43de2988bd79e4a2d8554515db8fb5aa36
2022-04-02 16:30:52 +05:30
MarcoFalke
6270a09847
Merge #17691: doc: Add missed copyright headers
fac86ac7b3ceac2f884412c7a9f4bd5bab5e3916 scripted-diff: Add missed copyright headers (Hennadii Stepanov)
6fde9d5e47fc9a1042b3fb68031eab5bf55e508d script: Update EXLUDE list in copyright_header.py (Hennadii Stepanov)
1998152f15fd2b0e83f5068c375a34feaf73db8c script: Add empty line after C++ copyright (Hennadii Stepanov)
071f2fc204f542c5a287ca8835115a2ee0bf2f50 script: Add ability to insert copyright to *.sh (Hennadii Stepanov)

Pull request description:

  This PR improves `contrib/devtools/copyright_header.py` script and adds copyright headers to the files in `src` and `test` directories with two exceptions:
  - [`src/reverse_iterator.h`](https://github.com/bitcoin/bitcoin/blob/master/src/reverse_iterator.h) (added to exceptions)
  - [`src/test/fuzz/FuzzedDataProvider.h`](https://github.com/bitcoin/bitcoin/blob/master/src/test/fuzz/FuzzedDataProvider.h) (added to exceptions)

  On master 5622d8f3156a293e61d0964c33d4b21d8c9fd5e0:
  ```
  $ ./contrib/devtools/copyright_header.py report . | grep zero
    25 with zero copyrights
  ```

  With this PR:
  ```
  $ ./contrib/devtools/copyright_header.py report . | grep zero
     2 with zero copyrights
  ```

  ~I am uncertain about our copyright policy with `build_msvc` and `contrib` directories content, so they are out of scope of this PR.~

ACKs for top commit:
  MarcoFalke:
    ACK fac86ac7b3ceac2f884412c7a9f4bd5bab5e3916

Tree-SHA512: d7832c4a7a1a3b7806119775b40ec35d7982f49ff0e6199b8cee4c0e0a36e68d51728b6ee9924b1c161df4bc6105bd93391b79d42914357fa522f499cb113fa8
2022-04-02 09:19:20 +05:30
Kittywhiskers Van Gogh
8f14c3e2e2 fuzz: add gitlab build support, use clang 2022-03-17 19:23:55 +05:30
fanquake
d9ec960820 Merge bitcoin/bitcoin#22516: macdeploy: alternative info to download the macOS SDK
f8f772dc495f7d3ee02d75d42ee22ceb90c3a693 macdeploy: alternative info to download the macOS SDK (Antoine Poinsot)

Pull request description:

  The previous link wasn't accessible for me, this adds some instructions
  given to me by Hebasto on #bitcoin-core-builds as well as a shasum for
  the archive to quickly check the downloaded one is the right one before
  processing with the entire Guix build.

ACKs for top commit:
  fanquake:
    ACK f8f772dc495f7d3ee02d75d42ee22ceb90c3a693

Tree-SHA512: 620160b593ed8fa4ae4a748b8e72d67b93ff0ec9e6b8ef3c3ac5402c1c48ec0ac325a527b6278cdf84aaf51ba8194d4c366c412ffad141d0412add2710efcff5
2022-03-13 14:52:10 -05:00
Wladimir J. van der Laan
3d6cc69131 Merge #20333: build: remove native_biplist dependency
7087440894a9daa7de806c5aa42d83ad60759c65 depends: native_ds_store 1.3.0 (fanquake)

Pull request description:

  `ds_store` [now takes advantage](36fb607940) of Pythons ability to decode binary [plists](https://docs.python.org/3/library/plistlib.html) (since 3.4), so we can drop its biplist dependency.

  The call to `biplist.Data()` in `custom_dsstore.py` doesn't seem to do anything, and from what I can tell can just be removed. i.e:
  ```diff
  diff --git a/contrib/macdeploy/custom_dsstore.py b/contrib/macdeploy/custom_dsstore.py
  index dc1c1882d..e475bc6c3 100755
  --- a/contrib/macdeploy/custom_dsstore.py
  +++ b/contrib/macdeploy/custom_dsstore.py
  @@ -47,6 +47,7 @@ alias.volume.disk_image_alias.target.filename = package_name_ns + '.temp.dmg'
   alias.volume.disk_image_alias.target.carbon_path = 'Macintosh HD:Users:\x00bitcoinuser:\x00Documents:\x00bitcoin:\x00bitcoin:\x00' + package_name_ns + '.temp.dmg'
   alias.volume.disk_image_alias.target.posix_path = 'Users/bitcoinuser/Documents/bitcoin/bitcoin/' + package_name_ns + '.temp.dmg'
   alias.target.carbon_path = package_name_ns + ':.background:\x00background.tiff'
  +assert(biplist.Data(alias.to_bytes()) == alias.to_bytes())
   icvp['backgroundImageAlias'] = biplist.Data(alias.to_bytes())
   ds['.']['icvp'] = icvp
  ```

ACKs for top commit:
  laanwj:
    ACK 7087440894a9daa7de806c5aa42d83ad60759c65

Tree-SHA512: 8ba3cf561937efe4a3daae8b0cb4de3bf9e425b3a9244161b09d94ee2b1bd4c3e21315fa70e495b19a052aabdc1731b3b6f346b63272d72d2762ced83237d02f
2022-03-05 13:48:08 -06:00
MarcoFalke
54de28adce Merge #15354: doc: Add missing bitcoin-wallet tool manpages
fa1e281547 doc: Add missing wallet-tool manpages (MarcoFalke)
fa0fe3b8a4 contrib: Add missing wallet tool to gen-manpages.sh (MarcoFalke)

Pull request description:

Tree-SHA512: 8c5c7e98f634cb1c8b43ecc9a15f22df2f572f5d752fb20f09910fb0d31e74df8144c1833f54bb44ad53cb5ca166f7e896317951899d4b0aa05bd3262f66835c
2022-03-04 23:38:32 -05:00
fanquake
9509768939 Merge #16812: doc: Fix whitespace errs in .md files, bitcoin.conf, and Info.plist.in
6aab7649d30b19d136a27f1287fd2c8b00fb460c doc: Fix whitespace errs in .md files, bitcoin.conf, Info.plist.in, and find_bdb48.m4 (Jon Layton)

Pull request description:

  Although there is an existing `test/lint/lint-whitespace.sh` linter, it only prevents new errors from being introduced. This commit removes all existing whitespace errors from Core markdown files (skips `src/crypto/ctaes/`, `leveldb/`, and `doc/release-notes/`), `bitcoin.conf`, and `Info.plist.in`.

  Further formatting could be done on the markdown documents, but seeing as there several coexisting styles that break a few `markdownlint` rules, a first step would be to define and add a linter to Travis. For now, the small fix is made.

ACKs for top commit:
  fanquake:
    ACK 6aab7649d30b19d136a27f1287fd2c8b00fb460c - Thanks for following up. Hopefully we now never have to deal with whitespace again.

Tree-SHA512: 810cc31ae4364b2dedf85783e67315d7b4e11589e4b32c599606e1b1ba8de0663bcae9ddb1bd8c9762a3636a2d65bdcd64ec22d2e90943f374a0c9574b77ca23
2022-01-20 13:09:17 -05:00
PastaPastaPasta
098209704b
optimize: adjust ci dockerfile for faster building (#4641)
* optimize: adjust ci dockerfile for faster building

* remove old comment

Co-authored-by: UdjinM6 <UdjinM6@users.noreply.github.com>

* remove unneeded semicolons

Co-authored-by: UdjinM6 <UdjinM6@users.noreply.github.com>

* format: make each installed package it's own line to minimize conflicts

* sort the installed packages (and a fix)

Signed-off-by: Pasta <pasta@dashboost.org>

Co-authored-by: UdjinM6 <UdjinM6@users.noreply.github.com>
2021-12-30 21:44:13 +03:00
PastaPastaPasta
06998263bb
Optimize linters (#4637)
* optimize: somehow optimize circular-dependencies.py

Signed-off-by: pasta <pasta@dashboost.org>

* optimize: use parallel if available to lint in parallel

Signed-off-by: pasta <pasta@dashboost.org>

* suggestions

* more suggestions

Co-authored-by: UdjinM6 <UdjinM6@users.noreply.github.com>
2021-12-30 19:34:36 +03:00
PastaPastaPasta
f503a7edc1
refactor: Fix warnings from cppcheck (#4625)
* src/evo/evodb.cpp:57:29: warning: Assert statement calls a function which may have desired side effects: 'IsClean'. [assertWithSideEffect]

* src/llmq/quorums.cpp:635:37: note: Null pointer dereference

src/llmq/quorums.cpp:635:37: warning: Either the condition 'pFrom==nullptr' is redundant or there is possible null pointer dereference: pFrom. [nullPointerRedundantCheck]
src/llmq/quorums.cpp:636:81: note: Assuming that condition 'pFrom==nullptr' is not redundant

* fix a bunch of cppcheck warnings

* cppcheck: run on many more files. Enable all checks except a few ignored ones.

ignored
```
    "Consider using std::transform algorithm instead of a raw loop."
    "Consider using std::accumulate algorithm instead of a raw loop."
```

* ci: build specific version of cppcheck instead of install from apt

* ci: use cppcheck 2.4, remove commented out line, fix symlink

cppcheck 2.6 is latest, however causes issues
```
src/spork.cpp:135:51: warning: Analysis failed. If the code is valid then please report this failure. [cppcheckError]
```
cppcheck 2.5 appears to get into an infinite loop

* no need to check presence before insertion

* use if-init, remove redundant check

* remove redundant check

* don't remove cmake? fix macOs depends build?

* cppcheck: one per line, alphabetize

* remove duplicate cmake install
2021-12-29 00:54:50 +03:00
Munkybooty
926d4a774f
lint: Fix typos flagged by codespell (#4639) 2021-12-29 00:45:54 +03:00
Kittywhiskers Van Gogh
58e4cca82b docs: introduce documentation and usage for containers 2021-12-21 12:43:48 +05:30
Kittywhiskers Van Gogh
9bedfa9838 build: add dash minimal development environment container
Co-authored-by: UdjinM6 <UdjinM6@users.noreply.github.com>
2021-12-21 12:43:37 +05:30
Kittywhiskers Van Gogh
1c188bcd47 build: relocate *net deployment logic to their own directories 2021-12-21 12:41:00 +05:30
Kittywhiskers Van Gogh
607707f9f9 build: relocate CI docker container to dedicated directory 2021-12-19 14:04:13 +05:30
UdjinM6
5502ce2705
Merge pull request #4602 from PastaPastaPasta/drop-unused-boost-libs
build: remove unused boost libraries
2021-12-11 23:02:44 +03:00
Pasta
4176234dea
remove references to libboost-system-dev and libboost-chrono-dev 2021-12-06 13:05:32 -05:00
UdjinM6
af68eeca22
Drop arm-linux-gnueabihf from releases 2021-12-04 00:51:20 +03:00
MarcoFalke
6c797b13e8
partial merge #21036: gitian: Bump descriptors to Focal for 22.0
2ecaf214331b506ebfac4f4922241744357d652b gitian: remove execstack workaround for ricv64 & powerpc64le (fanquake)
5baff2b31840bdbc465f55b875aa6e9480288215 build: use focal in gitian descriptors (fanquake)

Pull request description:

  This PR changes the gitian descriptors to use Ubuntu Focal (20.04), over Bionic (18.04), moving from GCC 7.5 to GCC 8.4 for native Linux builds, mingw-w64 GCC 7.3 to mingw-w64 GCC 9.3 for Windows builds, while continuing to use GCC 8.4 for all cross builds and Clang 8.0.0 for macOS builds.

  It also drops the `-Wl,-z,noexecstack` workaround we've been using for the riscv64 and powerpc64le hosts, as it's no-longer needed. One new package is installed in the osx build, `libtinfo5`, as libtinfo5.so is required by our downloaded Clang 8.

  A bump to Focal will at least be required if we want to update to a newer Qt (5.15, #19716) for 22.0, as we need a newer version of [`g++-mingw-w64`](https://packages.ubuntu.com/focal/g++-mingw-w64-x86-64) and the [`mingw-w64`](https://mingw-w64.org/doku.php) headers. This can still be done while continuing to use GCC 8.4 for Linux builds (see below), however the newer `g++-mingw-w64` will be based off of GCC 9.3.

  **Some considerations**

  GCC 9 is affected by #20005 "memcmp with constants that contain zero bytes are broken in GCC", and the newer `g++-mingw-w64` will be based off of GCC 9.3.

  The `--no-*` variants of the Windows linker flags (i.e `--no-dynamicbase`) we use to [test our `security-check.py` script](16b784d953/contrib/devtools/test-security-check.py (L53)) are not patched into the mingw binutils in Focal (they have been re-added in Groovy (20.10)). This isn't currently an issue, however, we might add a call to `test-security-check` for Guix (#20980), and if we wanted to do the same for gitian, it would not work. Note how it's quite "easy" for us to apply the `--no-*` variant patch to our Guix build; it would be quite a bit harder to do in Gitian.

  Gitian Builds @ 2ecaf214331b506ebfac4f4922241744357d652b

  #### Linux
  ```bash
  8882ea78486fbae4fac574b9089eb1107c6372d0dd7dfcda4f0f930576f9d6c1  bitcoin-2ecaf214331b-aarch64-linux-gnu-debug.tar.gz
  50a9e30943b4eee5163edff3331241e745ff32a2c4463c21a6fdc5986e2d0383  bitcoin-2ecaf214331b-aarch64-linux-gnu.tar.gz
  ec4e55a447fddf033fee33cd5f22bfeda3c3612f059194bcf6238859f7989d7a  bitcoin-2ecaf214331b-arm-linux-gnueabihf-debug.tar.gz
  444fe1b3b933c00bcbd4a9d86888cff3b61c1215b1debccd2843e842d1224777  bitcoin-2ecaf214331b-arm-linux-gnueabihf.tar.gz
  88e486ff465980dc1a4aab9687d142ec6f727ed2c52cf539f69db2877dee83b2  bitcoin-2ecaf214331b-powerpc64-linux-gnu-debug.tar.gz
  66144ac264c65cada9d86446e6026c85b04fb88198b8f41b42840f6031db3e6c  bitcoin-2ecaf214331b-powerpc64-linux-gnu.tar.gz
  34bcc13d78d929d575e34e77a6672f23ca7ea23230b28ec2eed563889352ba86  bitcoin-2ecaf214331b-powerpc64le-linux-gnu-debug.tar.gz
  b4c5f959664f3063df4330edfe343c17120eb6b556ee1c15c4aeb2c1c54ffd49  bitcoin-2ecaf214331b-powerpc64le-linux-gnu.tar.gz
  918fa72ab6f6ebce4e9663c93f72fe26651c260477cbb54749f7eb61438b5cc1  bitcoin-2ecaf214331b-riscv64-linux-gnu-debug.tar.gz
  f704f9f8c053ffe37d854e2e81e0f4c0614c435dad7f5d82518c681b73a76ae6  bitcoin-2ecaf214331b-riscv64-linux-gnu.tar.gz
  b59e3a62f1df9d79f30e916b3c9655f654036fe3a420040c53acc8dd9f4162c5  bitcoin-2ecaf214331b-x86_64-linux-gnu-debug.tar.gz
  a4dc9ca877cc97544e65db11be38406d16f15d74fcdcd2318bb92474729bc60d  bitcoin-2ecaf214331b-x86_64-linux-gnu.tar.gz
  b40ba2d5da498330ade92a4ccebcceb1452b94c8ffeacb336f87e93b5c88d8af  src/bitcoin-2ecaf214331b.tar.gz
  af6ebc91147778e4e6705eade62608dde4d6e60522d79087fa9129bdb7c01199  bitcoin-core-linux-22-res.yml
  ```

  #### Windows
  ```bash
  121a3970a6911cb8c453b2ce37d03f6cbb43333e29db8fa516c68563fb367f43  bitcoin-2ecaf214331b-win-unsigned.tar.gz
  6294e9efebe935092f9ba119dc60ad4094f18b51c4181324e54d3057524d6101  bitcoin-2ecaf214331b-win64-debug.zip
  5b5a236b63e67f5f6c07ad9aa716aa7b72fb63722c96798b332c6d164738f9cf  bitcoin-2ecaf214331b-win64-setup-unsigned.exe
  c1fa5894c5e02a201637567c80b9bde9024f44673dcd06fd4d489c1709179279  bitcoin-2ecaf214331b-win64.zip
  b40ba2d5da498330ade92a4ccebcceb1452b94c8ffeacb336f87e93b5c88d8af  src/bitcoin-2ecaf214331b.tar.gz
  665fd7eb61aed368150db58a254f15fb5efb51a4efa5abcc52571cb7a1a5de22  bitcoin-core-win-22-res.yml
  ```

  #### macOS
  ```bash
  6a1deae7662aa782baa82a42590f862c6bcdc4f4e38daa9b8c2a9eed1fbb5397  bitcoin-2ecaf214331b-osx-unsigned.dmg
  1ee843266e84928a4323fa255c833528c2617a2c9fd2f98fb26ba19bbfc1227b  bitcoin-2ecaf214331b-osx-unsigned.tar.gz
  097b64dadc167d8e5b733421bf1541a40760ad952990f7cf3f35adc6ae2616d0  bitcoin-2ecaf214331b-osx64.tar.gz
  b40ba2d5da498330ade92a4ccebcceb1452b94c8ffeacb336f87e93b5c88d8af  src/bitcoin-2ecaf214331b.tar.gz
  6e378fb543928e40c7119b96be6ff773d38506a9a888f8b02c7f1b8a0801a80e  bitcoin-core-osx-22-res.yml
  ```

ACKs for top commit:
  laanwj:
    Build script changes review ACK 2ecaf214331b506ebfac4f4922241744357d652b

Tree-SHA512: 975d5830b787d2e08988f43cbc6e839294171c1d94c8219636308b05f9b77041421612ae67be24a631674670cfc9c2d96d8177f2b3158a78fc3deea19631febf
2021-12-03 18:13:02 +03:00
Wladimir J. van der Laan
09199e8415
partial merge #20318: build: Ensure source tarball has leading directory name
faa2f06f5eaf8578873495f44603ee74d7a1abf4 scripted-diff: [build] Ensure source tarball has leading directory name (MarcoFalke)

Pull request description:

  This has been fixed in 0.20, so it needs to be fixed on master as well to avoid a regression

  #18945

ACKs for top commit:
  laanwj:
    ACK faa2f06f5eaf8578873495f44603ee74d7a1abf4
  hebasto:
    ACK faa2f06f5eaf8578873495f44603ee74d7a1abf4, tested gitian builds only.
  promag:
    ACK faa2f06f5eaf8578873495f44603ee74d7a1abf4.

Tree-SHA512: e3b025c29c45b025002abc35262bb5d771f6cbd807f1c256c477c243685e93cd43ad9f642b38e3cf218590912abe6ea0ddfec3bfbef36f99080aad74ed6cc0af
2021-12-03 18:13:02 +03:00
fanquake
2899ea1d89
Merge #19622: build: Drop ancient hack in gitian-linux descriptor
33a84e8f405ed6dd8885419cef305b4e6c7a428a build: Update and sort package list in gitian-linux.yml (Hennadii Stepanov)
95051682bedc2ef1076af5ede5bd56ca243279e3 build: Drop old hack which is unneeded now (Hennadii Stepanov)

Pull request description:

  The hack was aimed to fix an issue in Ubuntu Trusty 14.04 (see #8188).
  The current hack implementation was added in #8315.

  On master (8db23349fe9b512e6801d59d17052c5a7a1c64df) this hack is effectively noop, and it is no longer needed.

  I see this PR as a step to removing `libfaketime` from gitian builds.

ACKs for top commit:
  dongcarl:
    tACK 33a84e8f405e
  laanwj:
    Code review ACK 33a84e8f405ed6dd8885419cef305b4e6c7a428a

Tree-SHA512: 90036c555a500649ccc3d108bf11f09a9cfd2c92c0b598f7e0c0df63a713ae7abaf78f350b68c025470619c967223f45f6a235ad37a6ce1d1a0341ed34963ba0
2021-12-03 18:13:02 +03:00
fanquake
c0572028e3
partial merge #18741: guix: Make source tarball using git-archive
bfe1ba2f5b36056e0c41edf8206b93d3d83098df rel-builds: Specify core.abbrev for git-rev-parse (Carl Dong)
27e63e01cce368d67092de8f0c736927d6f6aa69 build: Accomodate makensis v2.x (Carl Dong)
1f2c39a30e0f82046c7aecddfda3eb99cb536816 guix: Remove logical cores requirement (Carl Dong)
a4f6ffa71e335d4b2a6bf525b7f416968f9cd9f7 lint: Also enable source statements for non-gitian (Carl Dong)
d256f91cb1b0d6ff5170106b99b0266cbe51f5a2 rel-builds: Directly deploy win installer to OUTDIR (Carl Dong)
fa791da02f9684e3fd554b687fb692ae6a23d65a nsis: Specify OutFile path only once (Carl Dong)
14701604d0904bc5bbf1c67de08f8ee6d3215523 guix: Expose GIT_COMMON_DIR in container as readonly (Carl Dong)
f5a6ac4f48b18f93050d77bcb23f9cf45ec34647 guix: Make source tarball using git-archive (Carl Dong)
395c1137f630dc495ffb2752a23bc1dfd470ee53 gitian: Limit sourced script to just assignments (Carl Dong)

Pull request description:

  Based on: #18556
  Related: https://github.com/bitcoin/bitcoin/pull/17595#discussion_r399728721

ACKs for top commit:
  fanquake:
    ACK bfe1ba2f5b36056e0c41edf8206b93d3d83098df - I agree with Carl, and am going to merge this. I'd like for Linux Guix builds to be working again, and we can rebase #18818.

Tree-SHA512: c87ada7e3de17ca0b692a91029b86573442ded5780fc081c214773f6b374a0cdbeaf6f6898c36669c2e247ee32aa7f82defb1180f8decac52c65f0c140f18674
2021-12-03 18:13:01 +03:00
fanquake
a0d3f37d3a
Merge #18556: build: Drop make dist in gitian builds
2aa48edec0101f8a77a2189244fc62722ff7a123 refactor: Drop unused ${WRAP_DIR}/${HOST} directory (Hennadii Stepanov)
1362be044724bb49d785ca2e296a3b43343c1690 build: Drop make dist in gitian builds (Hennadii Stepanov)

Pull request description:

  After the merge of #18331, the packaged source tarball is created by `git archive`, but the binaries are built from another one which is made by `make dist`.

  With this PR the only source tarball, created by `git archive`, is used both for binaries building and for packaging to users.

  Close #16588.
  Close #18547.

  As a good side-effect, #18349 becomes redundant.

  **Change in behavior**

  The following variables 1b151e3ffc/configure.ac (L2-L6)

  are no longer used for naming of directories and tarballs.

  Instead of them the gitian descriptors use a git tag (if available) or a commit hash.

  ---

  Also a small refactor commit picked from #18404.

ACKs for top commit:
  dongcarl:
    ACK 2aa48edec0101f8a77a2189244fc62722ff7a123
  MarcoFalke:
    ACK 2aa48edec0101f8a77a2189244fc62722ff7a123
  fanquake:
    ACK 2aa48edec0101f8a77a2189244fc62722ff7a123 - I've had a quick look over this, and don't want to block merging if this actually gets as closer to finally having this all sorted out. Obviously we've still got #18741, and after speaking to Carl this morning, there will likely be even more changes after that (not Guix specific).

Tree-SHA512: d3b16f87e48d1790a3264940c28acd5d881bfd10f3ce94fb0c8a6af76d8039289d01e0cd4972adac49ae24362857251f6c1e5e09e3e9fbf636c10708b4015a7c
2021-12-03 18:13:01 +03:00