Commit Graph

1082 Commits

Author SHA1 Message Date
UdjinM6
ea957a73a0
Merge pull request #4845 from Munkybooty/backports-0.20-pr3
Backports 0.20 pr3
2022-08-29 12:41:20 +03:00
MarcoFalke
2b035a75f2 Merge #17455: tests: Update valgrind suppressions
d604b4cc8c112a38976c4662cbdc3217a0e5b370 tests: Update valgrind suppressions (practicalswift)

Pull request description:

  Update `valgrind` suppressions.

  To test this PR:

  ```
  $ valgrind --suppressions=contrib/valgrind.supp src/test/test_bitcoin
  $ valgrind --suppressions=contrib/valgrind.supp src/bench/bench_bitcoin -evals=1 \
        -scaling=0.0
  ```

Top commit has no ACKs.

Tree-SHA512: 79cb318b5b9171e74d0bd0b89cc688ad4531b134182b06c2942c46058c19b45723c391b781e8ccd157a14fbf6a14588764c7728c5506c73ae237dde9f44db2f6
2022-08-24 14:29:45 -04:00
Kittywhiskers Van Gogh
6699b0228f
docker: fix gitian support, optimize container layer count and improve script consistency (#4983)
* contrib: set the working directory to /src/dash to allow for cloning gitian dependencies

* contrib: place the home directory inside /home instead of root

* contrib: add notes about sharing ccache across the network

* contrib: chown based on the (u/g)id env vars instead of the associated username

* contrib: reduce layer count by reducing run invocations

* contrib: develop container cleanup and maintenance

- add apt-cacher-ng, gpg, lsb-release, screen as a package dependencies
- reorder packages in alphabetical order
- correct documentation
- create and add user to the docker group to satisfy Gitian's needs
- reduce the number of RUN calls to reduce layer count
2022-08-22 22:42:36 +03:00
UdjinM6
8a9d0c5fe8
Merge pull request #4978 from UdjinM6/bp_macos_sig
backport: macos code signing
2022-08-17 19:19:02 +03:00
UdjinM6
00335a0282
bump signapple to the latest commit used in bitcoin atm
see 7a0b129c41/contrib/guix/manifest.scm (L502)
2022-08-17 18:35:47 +03:00
fanquake
6b615cfc67
Partial merge bitcoin/bitcoin#22190: Use latest signapple commit
683d197970a533690ca1bd4d06d021900e87cb8b Use latest signapple commit (Andrew Chow)

Pull request description:

  Update gitian and guix to use the same latest signapple commit.

  Also changed guix to use the actual repo. The changes from the fork were incorporated upstream.

ACKs for top commit:
  fanquake:
    ACK 683d197970a533690ca1bd4d06d021900e87cb8b - sanity checked that the updated package is built:

Tree-SHA512: a4981f8bbe33e6c5654632bc9b9f6f2f1e675741a19ac7296205e370f1e64a747101ecb632e0cc82a0134e4c2e9ce47b3f7b4d8c8f75f0f06dd069c078303759
2022-08-17 18:30:46 +03:00
Wladimir J. van der Laan
c57de2596a
Merge #20880: gitian: Use custom MacOS code signing tool
2c403279e2f0f7c8c27c56d4e7b0573c59571f0a gitian: Remove codesign_allocate and pagestuff from MacOS build (Andrew Chow)
f55eed251488d70d5e2e3a2965a4f8ec0c476853 gitian: use signapple to create the MacOS code signature (Andrew Chow)
95b06d21852b28712db6c710e420a58bdc1a0944 gitian: use signapple to apply the MacOS code signature (Andrew Chow)
42bb1ea363286b088257cabccb686ef1887c1d3b gitian: install signapple in gitian-osx-signer.yml (Andrew Chow)

Pull request description:

  The MacOS code signing issues that were encountered during the 0.21.0 release cycle have shown that it is necessary for us to use a code signing tool for which the source code is available and modifiable by us. Given that there appears to not be such a tool available, I have written such a tool, [signapple](https://github.com/achow101/signapple), that we can use. This tool is able to create a valid MacOS code signature, detach it in a way that we were doing previously, and attach it to the unsigned binary. This tool can also verify that the signature is correct.

  This PR implements the usage of that tool in the gitian build for the code signed MacOS binary. The code signer will use this tool to create the detached signature. Gitian builders will use this tool to apply the detached signature. The `gitian-osx-signer.yml` descriptor has been modified to install this tool so that the `detached-sig-apply.sh` script can use it. Additionally, the `codesign_allocate` and `pagestuff` tools are no longer necessary so they are no longer added to the tarball used in code signing. Lastly, both the `detached-sig-create.sh` and `detached-sig-apply.sh` scripts are made to be significantly less complex and to not do unexpected things such as unpacking an already unpacked tarball.

  The detached code signature that signapple creates is almost identical to that which we were previously creating. The only difference is that the cpu architecture name is included in the extension (e.g. we have `bitcoin-qt.x86_64sign` instead of `bitcoin-qt.sign`). This was done in order to support signing universal binaries which we may want to do in the future. However signapple can still apply existing code signatures as it will accept the `.sign` extension. If it is desired, it can be modified to produce signatures with just the `.sign` extension. However I do not think it is necessary to maintain compatibility with the old process.

ACKs for top commit:
  laanwj:
    Code review ACK 2c403279e2f0f7c8c27c56d4e7b0573c59571f0a

Tree-SHA512: 2a0e01e9133f8859b9de26e7e8fe1d2610d2cbdee2845e6008b12c083c7e3622cbb2d9b83c50a269e2c3074ab95914a8225d3cd4108017f58b77a62bf10951e0
2022-08-17 18:30:05 +03:00
UdjinM6
fc01d76fa9
fix(gitian): Fetch tags while cloning the repo via gbuild (#4976)
Should fix version numbers for releases/candidates
2022-08-17 17:37:07 +03:00
PastaPastaPasta
f83d857e36
Merge pull request #4478 from kittywhiskers/undashify
revert #3036, #2911, #1879, #1432: revert dashification of filenames
2022-08-11 12:34:55 -04:00
PastaPastaPasta
afbc817220
refactor/chore: update cppcheck to 2.8 with needed refactoring (#4926)
* refactor/chore: update cppcheck to 2.8 with needed refactoring

* use probably invalid index for default

Co-authored-by: UdjinM6 <UdjinM6@users.noreply.github.com>

* trivial: rename skContributions -> m_sk_contributions and skContributions2 -> skContributions

Co-authored-by: UdjinM6 <UdjinM6@users.noreply.github.com>
2022-08-11 02:05:44 +03:00
Konstantin Akimov
137c4fc84c
Update hard coded seeds for v18.0.0-rc12 (#4953)
Mainnet: 1716101
    Testnet: 774784
2022-08-11 01:57:13 +03:00
Kittywhiskers Van Gogh
c587212f8c partial revert dash#2911: s/dash-config/bitcoin-config/g 2022-08-09 14:16:28 +05:30
PastaPastaPasta
6c0a63360f
Merge pull request #4882 from kittywhiskers/xliff_update
backport: bitcoin#21694 (Use XLIFF file to provide more context to Transifex)
2022-07-13 08:27:34 -04:00
PastaPastaPasta
d89de3a2fa
Merge pull request #4866 from Munkybooty/backports-0.20-pr4
Backports 0.20 pr4
2022-06-30 10:08:59 -05:00
Wladimir J. van der Laan
33c473a8e3
Merge #18796: scripts: security-check.py refactors
eacedfb0230978748cbcfb13817fed7e7c756ba7 scripts: add additional type annotations to security-check.py (fanquake)
83d063e9541cc9ea41ea86919eb9435c73efb14e scripts: add run_command to security-check.py (fanquake)
13f606b4f940e5820ff21ea62fc27a5a91774b05 scripts: remove NONFATAL from security-check.py (fanquake)
061acf62a15ad3dbb9f055b7c2569b9832ed623a scripts: no-longer check for 32 bit windows in security-check.py (fanquake)

Pull request description:

  * Remove 32-bit Windows checks.
  * Remove NONFATAL checking. Added in #8249, however unused since #13764.
  * Add `run_command` to de-duplicate all of the subprocess calls. Mentioned in #18713.
  * Add additional type annotations.
  * Print stderr when there is an issue running a command.

ACKs for top commit:
  laanwj:
    ACK eacedfb0230978748cbcfb13817fed7e7c756ba7

Tree-SHA512: 69a7ccfdf346ee202b3e8f940634c5daed1d2b5a5d15ac9800252866ba3284ec66e391a66a0b341f5a4e5e8482fe1b614d4671e8e766112ff059405081184a85
2022-06-18 22:14:03 -07:00
fanquake
f7a94b58d4
Merge #18629: scripts: add PE .reloc section check to security-check.py
3e38023af724a76972d39cbccfb0bba4c54a0323 scripts: add PE .reloc section check to security-check.py (fanquake)

Pull request description:

  The `ld` in binutils has historically had a few issues with PE binaries, there's a good summary in this [thread](https://sourceware.org/bugzilla/show_bug.cgi?id=19011).

  One issue in particular was `ld` stripping the `.reloc` section out of PE binaries, even though it's required for functioning ASLR. This was [reported by a Tor developer in 2014](https://sourceware.org/bugzilla/show_bug.cgi?id=17321) and they have been patching their [own binutils](https://gitweb.torproject.org/builders/tor-browser-build.git/tree/projects/binutils) ever since. However their patch only made it into binutils at the [start of this year](https://sourceware.org/git/gitweb.cgi?p=binutils-gdb.git;a=commit;h=dc9bd8c92af67947db44b3cb428c050259b15cd0). It adds an `--enable-reloc-section` flag, which is turned on by default if you are using `--dynamic-base`. In the mean time this issue has also been worked around by other projects, such as FFmpeg, see [this commit](91b668acd6).

  I have checked our recent supported Windows release binaries, and they do contain a `.reloc` section. From what I understand, we are using all the right compile/linker flags, including `-pie` & `-fPIE`, and have never run into the crashing/entrypoint issues that other projects might have seen.

  One other thing worth noting here, it how Debian/Ubuntu patch the binutils that they distribute, because that's what we end up using in our gitian builds.

  In the binutils-mingw-w64 in Bionic (18.04), which we currently use in gitian, PE hardening options/security flags are enabled by default. See the [changelog](https://changelogs.ubuntu.com/changelogs/pool/universe/b/binutils-mingw-w64/binutils-mingw-w64_8ubuntu1/changelog) and the [relevant commit](452b3013b8).

  However in Focal (20.04), this has now been reversed. PE hardening options are no-longer the default. See the [changelog](https://changelogs.ubuntu.com/changelogs/pool/universe/b/binutils-mingw-w64/binutils-mingw-w64_8.8/changelog) and [relevant commit](7bd8b2fbc2), which cites same .reloc issue mentioned here.

  Given that we explicitly specify/opt-in to everything that we want to use, the defaults aren't necessarily an issue for us. However I think it highlights the importance of continuing to be explicit about what we want, and not falling-back or relying on upstream.

  This was also prompted by the possibility of us doing link time garbage collection, see #18579 & #18605. It seemed some sanity checks would be worthwhile in-case the linker goes haywire while garbage collecting.

  I think Guix is going to bring great benefits when dealing with these kinds of issues. Carl you might have something to say in that regard.

ACKs for top commit:
  dongcarl:
    ACK 3e38023af724a76972d39cbccfb0bba4c54a0323

Tree-SHA512: af14d63bdb334bde548dd7de3e0946556b7e2598d817b56eb4e75b3f56c705c26aa85dd9783134c4b6a7aeb7cb4de567eed996e94d533d31511f57ed332287da
2022-06-18 22:14:03 -07:00
fanquake
bd7b5a3ad4
Merge #18713: scripts: Add MACHO stack canary check to security-check.py
8334ee31f868f0f9baf0920d14d20174ed889dbe scripts: add MACHO LAZY_BINDINGS test to test-security-check.py (fanquake)
7b99c7454cdb74cd9cd7a5eedc2fb9d0a19df456 scripts: add MACHO Canary check to security-check.py (fanquake)

Pull request description:

  7b99c7454cdb74cd9cd7a5eedc2fb9d0a19df456 uses `otool -Iv` to check for `___stack_chk_fail` in the macOS binaries. Similar to the [ELF check](https://github.com/bitcoin/bitcoin/blob/master/contrib/devtools/security-check.py#L105). Note that looking for a triple underscore prefixed function (as opposed to two for ELF) is correct for the macOS binaries. i.e:
  ```bash
  otool -Iv bitcoind | grep chk
  0x00000001006715b8   509 ___memcpy_chk
  0x00000001006715be   510 ___snprintf_chk
  0x00000001006715c4   511 ___sprintf_chk
  0x00000001006715ca   512 ___stack_chk_fail
  0x00000001006715d6   517 ___vsnprintf_chk
  0x0000000100787898   513 ___stack_chk_guard
  ```

  8334ee31f868f0f9baf0920d14d20174ed889dbe is a follow up to #18295 and adds test cases to `test-security-check.py` that for some reason I didn't add at the time. I'll sort out #18434 so that we can run these tests in the CI.

ACKs for top commit:
  practicalswift:
    ACK 8334ee31f868f0f9baf0920d14d20174ed889dbe: Mitigations are important. Important things are worth asserting :)
  jonasschnelli:
    utACK 8334ee31f868f0f9baf0920d14d20174ed889dbe.

Tree-SHA512: 1aa5ded34bbd187eddb112b27278deb328bfc21ac82316b20fab6ad894f223b239a76b53dab0ac1770d194c1760fcc40d4da91ec09959ba4fc8eadedb173936a
2022-06-18 22:14:03 -07:00
fanquake
67ceda1b5a
Merge #18295: scripts: add MACHO lazy bindings check to security-check.py
5ca90f8b598978437340bb8467f527b9edfb2bbf scripts: add MACHO lazy bindings check to security-check.py (fanquake)

Pull request description:

  This is a slightly belated follow up to #17686 and some discussion with Cory. It's not entirely clear if we should make this change due to the way the macOS dynamic loader appears to work. However I'm opening this for some discussion. Also related to #17768.

  #### Issue:
  [`LD64`](https://opensource.apple.com/source/ld64/) doesn't set the [MH_BINDATLOAD](https://opensource.apple.com/source/xnu/xnu-6153.11.26/EXTERNAL_HEADERS/mach-o/loader.h.auto.html) bit in the header of MACHO executables, when building with `-bind_at_load`. This is in contradiction to the [documentation](https://opensource.apple.com/source/ld64/ld64-450.3/doc/man/man1/ld.1.auto.html):
  ```bash
  -bind_at_load
       Sets a bit in the mach header of the resulting binary which tells dyld to
       bind all symbols when the binary is loaded, rather than lazily.
  ```

  The [`ld` in Apples cctools](https://opensource.apple.com/source/cctools/cctools-927.0.2/ld/layout.c.auto.html) does set the bit, however the [cctools-port](https://github.com/tpoechtrager/cctools-port/) that we use for release builds, bundles `LD64`.

  However; even if the linker hasn't set that bit, the dynamic loader ([`dyld`](https://opensource.apple.com/source/dyld/)) doesn't seem to ever check for it, and from what I understand, it looks at a different part of the header when determining whether to lazily load symbols.

  Note that our release binaries are currently working as expected, and no lazy loading occurs.

  #### Example:

  Using a small program, we can observe the behaviour of the dynamic loader.

  Conducted using:
  ```bash
  clang++ --version
  Apple clang version 11.0.0 (clang-1100.0.33.17)
  Target: x86_64-apple-darwin18.7.0

  ld -v
  @(#)PROGRAM:ld  PROJECT:ld64-530
  BUILD 18:57:17 Dec 13 2019
  LTO support using: LLVM version 11.0.0, (clang-1100.0.33.17) (static support for 23, runtime is 23)
  TAPI support using: Apple TAPI version 11.0.0 (tapi-1100.0.11)
  ```

  ```cpp
  #include <iostream>
  int main() {
  	std::cout << "Hello World!\n";
  	return 0;
  }
  ```

  Compile and check the MACHO header:
  ```bash
  clang++ test.cpp -o test
  otool -vh test
  ...
  Mach header
        magic cputype cpusubtype  caps    filetype ncmds sizeofcmds      flags
  MH_MAGIC_64  X86_64        ALL LIB64     EXECUTE    16       1424   NOUNDEFS DYLDLINK TWOLEVEL WEAK_DEFINES BINDS_TO_WEAK PIE

  # Run and dump dynamic loader bindings:
  DYLD_PRINT_BINDINGS=1 DYLD_PRINT_TO_FILE=no_bind.txt ./test
  Hello World!
  ```

  Recompile with `-bind_at_load`. Note still no `BINDATLOAD` flag:
  ```bash
  clang++ test.cpp -o test -Wl,-bind_at_load
  otool -vh test
  Mach header
        magic cputype cpusubtype  caps    filetype ncmds sizeofcmds      flags
  MH_MAGIC_64  X86_64        ALL LIB64     EXECUTE    16       1424   NOUNDEFS DYLDLINK TWOLEVEL WEAK_DEFINES BINDS_TO_WEAK PIE
  ...
  DYLD_PRINT_BINDINGS=1 DYLD_PRINT_TO_FILE=bind.txt ./test
  Hello World!
  ```

  If we diff the outputs, you can see that `dyld` doesn't perform any lazy bindings when the binary is compiled with `-bind_at_load`, even if the `BINDATLOAD` flag is not set:
  ```diff
  @@ -1,11 +1,27 @@
  +dyld: bind: test:0x103EDF030 = libc++.1.dylib:__ZNKSt3__16locale9use_facetERNS0_2idE, *0x103EDF030 = 0x7FFF70C9FA58
  +dyld: bind: test:0x103EDF038 = libc++.1.dylib:__ZNKSt3__18ios_base6getlocEv, *0x103EDF038 = 0x7FFF70CA12C2
  +dyld: bind: test:0x103EDF068 = libc++.1.dylib:__ZNSt3__113basic_ostreamIcNS_11char_traitsIcEEE6sentryC1ERS3_, *0x103EDF068 = 0x7FFF70CA12B6
  +dyld: bind: test:0x103EDF070 = libc++.1.dylib:__ZNSt3__113basic_ostreamIcNS_11char_traitsIcEEE6sentryD1Ev, *0x103EDF070 = 0x7FFF70CA1528
  +dyld: bind: test:0x103EDF080 = libc++.1.dylib:__ZNSt3__16localeD1Ev, *0x103EDF080 = 0x7FFF70C9FAE6
  <trim>
  -dyld: lazy bind: test:0x10D4AC0C8 = libsystem_platform.dylib:_strlen, *0x10D4AC0C8 = 0x7FFF73C5C6E0
  -dyld: lazy bind: test:0x10D4AC068 = libc++.1.dylib:__ZNSt3__113basic_ostreamIcNS_11char_traitsIcEEE6sentryC1ERS3_, *0x10D4AC068 = 0x7FFF70CA12B6
  -dyld: lazy bind: test:0x10D4AC038 = libc++.1.dylib:__ZNKSt3__18ios_base6getlocEv, *0x10D4AC038 = 0x7FFF70CA12C2
  -dyld: lazy bind: test:0x10D4AC030 = libc++.1.dylib:__ZNKSt3__16locale9use_facetERNS0_2idE, *0x10D4AC030 = 0x7FFF70C9FA58
  -dyld: lazy bind: test:0x10D4AC080 = libc++.1.dylib:__ZNSt3__16localeD1Ev, *0x10D4AC080 = 0x7FFF70C9FAE6
  -dyld: lazy bind: test:0x10D4AC070 = libc++.1.dylib:__ZNSt3__113basic_ostreamIcNS_11char_traitsIcEEE6sentryD1Ev, *0x10D4AC070 = 0x7FFF70CA1528
  ```

  Note: `dyld` also has a `DYLD_BIND_AT_LAUNCH=1` environment variable, that when set, will force any lazy bindings to be non-lazy:
  ```bash
  dyld: forced lazy bind: test:0x10BEC8068 = libc++.1.dylib:__ZNSt3__113basic_ostream
  ```

  #### Thoughts:
  After looking at the dyld source, I can't find any checks for `MH_BINDATLOAD`. You can see the flags it does check for, such as MH_PIE or MH_BIND_TO_WEAK [here](https://opensource.apple.com/source/dyld/dyld-732.8/src/ImageLoaderMachO.cpp.auto.html).

  It seems that the lazy binding of any symbols depends on whether or not [lazy_bind_size](https://opensource.apple.com/source/xnu/xnu-6153.11.26/EXTERNAL_HEADERS/mach-o/loader.h.auto.html) from the `LC_DYLD_INFO_ONLY` load command is > 0. Which was mentioned in [#17686](https://github.com/bitcoin/bitcoin/pull/17686#issue-350216254).

  #### Changes:
  This PR is one of [Corys commits](7b6ba26178), that I've rebased and modified to make build. I've also included an addition to the `security-check.py` script to check for the flag.

  However, given the above, I'm not entirely sure this patch is the correct approach. If the linker no-longer inserts it, and the dynamic loader doesn't look for it, there might be little benefit to setting it. Or, maybe this is an oversight from Apple and needs some upstream discussion. Looking for some thoughts / Concept ACK/NACK.

  One alternate approach we could take is to drop the patch and modify security-check.py to look for `lazy_bind_size` == 0 in the `LC_DYLD_INFO_ONLY` load command, using `otool -l`.

ACKs for top commit:
  theuni:
    ACK 5ca90f8b598978437340bb8467f527b9edfb2bbf

Tree-SHA512: 444022ea9d19ed74dd06dc2ab3857a9c23fbc2f6475364e8552d761b712d684b3a7114d144f20de42328d1a99403b48667ba96885121392affb2e05b834b6e1c
2022-06-18 22:14:02 -07:00
Wladimir J. van der Laan
3c4982205c
Merge #18415: scripts: add MACHO tests to test-security-check.py
7142d50ac33e0ad7d24e49e04c1fc7e3e769ed46 scripts: rename test_64bit_PE to test_PE (fanquake)
edaca2dd123cef958699c07ab248cf0ffc71af07 scripts: add MACHO NX check to security-check.py (fanquake)
1a4e9f32efcc5f6a74290446dc58784fd85c7b31 scripts: add MACHO tests to test-security-check.py (fanquake)

Pull request description:

  Adds tests for the MACHO checks in security-check.py:
  ac579ada7e/contrib/devtools/security-check.py (L212-L214)

  I'm planning on following up with more checks in security-check.py, and corresponding tests in test-security-check.py.

  Note that you'll probably have to be on macOS to run them. You can run just this suite with `python3 test-security-check.py TestSecurityChecks.test_MACHO`.

ACKs for top commit:
  laanwj:
    ACK 7142d50ac33e0ad7d24e49e04c1fc7e3e769ed46

Tree-SHA512: ace3ca9f6df5d4fedd5988938fb7dc7563ec7dc587aa275f780b5f51e9b8d7d6f7768e0a1e05ce438510a07b8640aba92c76847b30c2990f46c66b78a0acf960
2022-06-18 22:14:02 -07:00
fanquake
1ff571fc2c Merge #17515: Remove straggling OpenSSL references from doc and build
ea3c7e585c382998212fd7f41114462a8168a734 test: Remove libssl-dev packages from CI scripts (Wladimir J. van der Laan)
7ea55264b9d60325bc7a5c15d78e9063de145970 test: remove lsan suppression for libcrypto (Wladimir J. van der Laan)
2d7066527a456f8e1f4f603fe104b0bd9d864559 build: remove libcrypto as internal dependency in libbitcoinconsensus.pc (Wladimir J. van der Laan)
278751ea11f2cfe68b0c98f504f65586720cb5a4 doc: Remove ssl as a required dependency from build-unix (Wladimir J. van der Laan)

Pull request description:

  Some doc and build cleanups following #17265.

  I intentionally left the libssl-dev install in `gitian-win-signer.yml`, as it's necessary for the ossl signer.

ACKs for top commit:
  MarcoFalke:
    ACK ea3c7e585c382998212fd7f41114462a8168a734 🗯
  jamesob:
    ACK ea3c7e585c
  practicalswift:
    ACK ea3c7e585c382998212fd7f41114462a8168a734 - nice!
  fanquake:
    ACK ea3c7e585c382998212fd7f41114462a8168a734 - thanks.

Tree-SHA512: 67ea35bdd6d6e512d69e6734713534c88cae033a2ed695677ea15c3e3d5ff570374e342775c88e60877fa43a19047853e7b2a433e2c9a4349a5c423726a7457e
2022-06-16 01:34:30 -04:00
Kittywhiskers Van Gogh
32b3c2d98a ci: add text extraction dependency to permit running translation routines 2022-06-11 21:03:04 +05:30
Wladimir J. van der Laan
a25ee06cc1 Merge #17787: scripts: add MACHO PIE check to security-check.py
7c9e821c4e6cb186208ead9c8df616d1f393a49a scripts: add MACHO NOUNDEFS check to security-check.py (fanquake)
4ca92dc6d3f3e487d63286d8871d1829b3d279ff scripts: add MACHO PIE check to security-check.py (fanquake)

Pull request description:

  This uses `otool -vh` to print the mach header and look for the `PIE` flag:
  ```bash
  otool -vh src/bitcoind
  Mach header
        magic cputype cpusubtype  caps    filetype ncmds sizeofcmds      flags
  MH_MAGIC_64  X86_64        ALL LIB64     EXECUTE    24       2544   NOUNDEFS DYLDLINK TWOLEVEL WEAK_DEFINES BINDS_TO_WEAK PIE
  ```

  From [`mach-o/loader.h`](https://opensource.apple.com/source/cctools/cctools-927.0.2/include/mach-o/loader.h.auto.html):
  ```c
  #define	MH_PIE 0x200000			/* When this bit is set, the OS will
  					   load the main executable at a
  					   random address.  Only used in
  					   MH_EXECUTE filetypes. */
  ```

ACKs for top commit:
  laanwj:
    code review ACK 7c9e821c4e6cb186208ead9c8df616d1f393a49a

Tree-SHA512: 5ba2f60440d0e31c70371a355c91ca4f723d80f7287d04e2098bf5b11892cc74216ff8f1454603c4db9675d4f7983614843b992b8dcfca0309aadf2aa7ab2e4b
2022-06-08 12:36:52 +07:00
PastaPastaPasta
d64b7229cd
chore: bump copyrights (#4873)
* chore: bump copyright in configure.ac

* chore: bump copyright via copyright_header.py

ran command `python3 contrib/devtools/copyright_header.py update .`
2022-06-08 02:36:46 +03:00
PastaPastaPasta
875305a901
Merge pull request #4839 from Munkybooty/backports-0.20-pr2
backport: v0.20 pr2
2022-06-01 13:53:54 -05:00
Wladimir J. van der Laan
37ef9bb465 Merge #17282: contrib: remove accounts from bash completion
dd94cc46e4377b14515be24894fd1cb42e233151 contrib: remove accounts from bash completion (fanquake)

Pull request description:

  Also removes `setgenerate`.

ACKs for top commit:
  jonasschnelli:
    utACK dd94cc46e4377b14515be24894fd1cb42e233151

Tree-SHA512: 218aa06841f4a347bb083c7f408c07fe80df7a30f7ce7a1126ec95971a9b66aff7c3c049d269da2c15f629bbde80e8c303bf6562bd5c1a36a6f6fd61ce9133e2
2022-05-31 12:06:32 -04:00
Kittywhiskers Van Gogh
3ba8431900 merge bitcoin#16899: UTXO snapshot creation (dumptxoutset)
Co-authored-by: UdjinM6 <UdjinM6@users.noreply.github.com>
2022-05-23 10:38:33 +05:30
strophy
ff2f768114
ci: fix docker context (#4816) 2022-04-28 11:35:50 -05:00
MarcoFalke
a3a7a22268
Merge #20223: build: Drop the leading 0 from the version number
8f7b93047581c67f2133cdb8c7845471de66c30f Drop the leading 0 from the version number (Andrew Chow)

Pull request description:

  Removes the leading 0 from the version number. The minor version, which we had been using as the major version, is now the major version. The revision, which we had been using as the minor version, is now the minor version. The revision number is dropped. The build number is promoted to being part of the version number. This also avoids issues where it was accidentally not included in the version number.

  The CLIENT_VERSION remains the same format as previous as previously, as the Major version was 0 so it never actually got included in it.

  The user agent string formatter is updated to follow this new versioning.

  ***

  Honestly I'm just tired of all of the people asking for "1.0" that maybe this'll shut them up. Skip the whole 1.0 thing and go straight to version 22.0!

  Also, this means that the terminology we commonly use lines up with how the variables are named. So major versions are actually bumping the major version number, etc.

ACKs for top commit:
  jnewbery:
    Code review ACK 8f7b930475
  MarcoFalke:
    review ACK 8f7b93047581c67f2133cdb8c7845471de66c30f 🎻

Tree-SHA512: b5c3fae14d4c0a9c0ab3b1db7c949ecc0ac3537646306b13d98dd0efc17c489cdd16d43f0a24aaa28e9c4a92ea360500e05480a335b03f9fb308010cdd93a436
2022-04-28 13:47:53 +03:00
PastaPastaPasta
b421cfacae
perf: enable more multi-threading and caching in linters (#4807)
* perf: enable more multi-threading and caching in linters

20s -> 6s

* ci: add multiprocess to ci dockerfile

* Update test/lint/lint-cppcheck-dash.sh

Co-authored-by: UdjinM6 <UdjinM6@users.noreply.github.com>

Co-authored-by: UdjinM6 <UdjinM6@users.noreply.github.com>
2022-04-27 21:14:40 +03:00
UdjinM6
c6e724bf21 partial bitcoin#22244: Add xkb version to symbol-check 2022-04-26 20:37:20 +05:30
Kittywhiskers Van Gogh
7b69a4cb42 merge bitcoin#21654: Make Qt rcc output always deterministic 2022-04-26 20:36:54 +05:30
Kittywhiskers Van Gogh
b9b9473f55 merge bitcoin#21655: No longer need to set QT_RCC_TEST=1 for determinism 2022-04-26 20:36:54 +05:30
Kittywhiskers Van Gogh
1763f727b5 merge bitcoin#21376: Qt 5.12.10 2022-04-26 00:54:10 +05:30
Kittywhiskers Van Gogh
43152b2b35
merge #17165: Remove BIP70 support (#4023)
* compat: remove bswap_* check on macOS

This was originally added in #9366 to fix the gui build, as
Protobuf would also define these macros. Now that we're no-longer
using Protobuf, remove the additional check.

* build: skip building OpenSSL lib_ssl

* build: remove OpenSSL from Qt build

More info available from:
https://doc.qt.io/qt-5/ssl.html#enabling-and-disabling-ssl-support

* build: remove EVP_MD_CTX_new detection

This was added in #9475 to fix LibreSSL compatibility for
BIP70, so is no longer required.

* build: remove SSL lib detection

* gui: update BIP70 support message

* build: remove BIP70 entries from macOS Info.plist

* gui: remove payment request file handling from OpenURI dialog

* gui: remove BIP70 Support

* build: remove protobuf from depends and contrib
2022-04-25 12:01:47 +03:00
ktechmidas
712f60dd21
Fix: Use github workspace for building image (#4775)
* Fix: Use github workspace for building image

* Fix: Ensure correct dir for docker-entrypoint.sh
2022-04-19 09:11:04 +03:00
UdjinM6
89f606a927
trivial: add some missing dashifications (#4772) 2022-04-19 09:09:42 +03:00
pasta
728699d2cc chore: bump version in gitian-descriptors and README.md 2022-04-16 07:59:23 -06:00
UdjinM6
903291788c
Merge pull request #4735 from kittywhiskers/translation
merge bitcoin#16244...18699: translation backports
2022-04-07 13:18:29 +03:00
UdjinM6
eb42d8eee4
Merge pull request #4749 from PastaPastaPasta/develop-trivial-2022-04-03-pr2
trivial backports 2022 04 03 pr2
2022-04-07 13:17:05 +03:00
Kittywhiskers Van Gogh
0f90ea72d3 merge bitcoin#15450: Create wallet menu option 2022-04-06 13:49:58 +05:30
W. J. van der Laan
6525ca37a7 Merge bitcoin/bitcoin#21709: doc: update reduce-memory.md and bitcoin.conf -maxconnections info
300234ab6698277fb261a87be72f89ef94d3697a doc: update bitcoin.conf maxconnections info (Jon Atack)
926827065ffa56b75e9261f63d49b924d4bced0f doc: update reduce-memory.md peer connections info (Jon Atack)

Pull request description:

  This patch updates the documentation in `doc/reduce-memory.md` and `share/examples/bitcoin.conf` regarding the peer connections limits and `-maxconnections`

ACKs for top commit:
  jarolrod:
    re-ACK 300234ab6698277fb261a87be72f89ef94d3697a
  laanwj:
    ACK 300234ab6698277fb261a87be72f89ef94d3697a
  prayank23:
    ACK 300234ab66

Tree-SHA512: 90f53626124afb50706e6a3b644bc7bb800bb7cf41ae7062c20c17b3d9bdf4a8d73b4cf188faec9113d772596f7e4bc36a4a69481cacb92cc55d5956181d0c31
2022-04-05 23:10:05 -05:00
fanquake
c841e6e66f Merge bitcoin/bitcoin#23598: contrib: Specify wb mode when creating mac sdk
655d52a0cd6b3df738dc9843e25a95b8930b4628 contrib: Specify wb mode when creating mac sdk (João Barbosa)

Pull request description:

  Fix the warning:
  ```
  ./contrib/macdeploy/gen-sdk:84: FutureWarning: GzipFile was opened for writing, but this will change in future Python releases.  Specify the mode argument for opening it for writing.
  ```

ACKs for top commit:
  fanquake:
    Tested ACK 655d52a0cd6b3df738dc9843e25a95b8930b4628 with Python 3.10.

Tree-SHA512: 095cd301f211531a8b8f50e7915fe13c7ab0b278fb23dc50a03625cfae9e2fd7a0d8c315fced4af3011552e3e455ce562b7f717a0ed096c4433ddcf24f22b2c9
2022-04-03 18:48:15 -05:00
PastaPastaPasta
e84bf45cef
Merge pull request #4744 from vijaydasmp/bp2003
Merge #17119,1683,17102,17134,17691,17992,18170,18382,16975,18472, 18486,17633,7833
2022-04-03 17:44:16 -05:00
PastaPastaPasta
279686f9ba
Merge pull request #4742 from vijaydasmp/bp2002
Merge #17154,16569,17351,13716,17084,18391,16821,17140
2022-04-03 17:33:44 -05:00
strophy
1fddc53f53
ci: build on-demand docker images for testing on devnets (#4667)
* ci: build alpha action

* ci: use self-hosted runner

* ci: build faster

* ci: multicore dash build

* chore: copy build, dockerize and push

* chore: dockerize build

* chore: remove unnecessary cachedir and checkout steps

* chore: disable qemu

* chore: remove buildx steps now included in AMI

* chore: move binaries in to place for second stage copy

* fix: runner using incorrect build driver

* chore: debug builder info

* chore: more detailed debug

* chore: install qemu and buildx in actions

* fix: indentation

* fix: remove debug code

* feat: multi-arch cross-compiling dockerfile

* chore: remove unnecessary docker layers

* chore: add debug output

* chore: dump context

* fix: use event inputs tag for docker_meta

* chore: dump context again

* fix: context reference syntax

* feat: attempt to use gitian builder

* chore: ff changes from other branches

* chore: disable macOS build

* Revert "chore: ff changes from other branches"

This reverts commit daece1c505.

* chore: restore GHA changes to fix incorrect merge

* chore: remove duplicate code

* chore: bump buildx version to latest

* chore: try to build without signing

* chore: setup tmate to find build

* chore: run tmate on failure

* fix: invalid path to binaries on copy

* chore: remove unnecessary GPG steps

* chore: replace strophy with dashpay

* fix: gha cache not providing speedup

* Update .github/workflows/release_alpha.yml

Co-authored-by: UdjinM6 <UdjinM6@users.noreply.github.com>
2022-04-03 17:24:06 -05:00
Wladimir J. van der Laan
cca430ba9a
Merge #16569: Increase init file stop timeout
7fb7acfc206b4bf8c296d72b66f3bd4fe342fd87 Set init stop timeout to 10 min (setpill)

Pull request description:

  `bitcoind` can take a long time to flush its db cache to disk upon
  shutdown. Systemd sends a `SIGKILL` after a timeout, causing unclean
  shutdowns and triggering a long "Rolling forward" at the next startup.
  Disabling the timeout should prevent this from happening, and does not
  break systemd's `restart` logic.

  Addresses #13736.

ACKs for top commit:
  instagibbs:
    utACK 7fb7acfc20

Tree-SHA512: 16e0ce5a9ecf0628f8d93d68db3f5a78ab36021d9bede05a90c84f144db2e87e17707a6eb910cb7c018c265ce2c81d43de2988bd79e4a2d8554515db8fb5aa36
2022-04-02 16:30:52 +05:30
MarcoFalke
6270a09847
Merge #17691: doc: Add missed copyright headers
fac86ac7b3ceac2f884412c7a9f4bd5bab5e3916 scripted-diff: Add missed copyright headers (Hennadii Stepanov)
6fde9d5e47fc9a1042b3fb68031eab5bf55e508d script: Update EXLUDE list in copyright_header.py (Hennadii Stepanov)
1998152f15fd2b0e83f5068c375a34feaf73db8c script: Add empty line after C++ copyright (Hennadii Stepanov)
071f2fc204f542c5a287ca8835115a2ee0bf2f50 script: Add ability to insert copyright to *.sh (Hennadii Stepanov)

Pull request description:

  This PR improves `contrib/devtools/copyright_header.py` script and adds copyright headers to the files in `src` and `test` directories with two exceptions:
  - [`src/reverse_iterator.h`](https://github.com/bitcoin/bitcoin/blob/master/src/reverse_iterator.h) (added to exceptions)
  - [`src/test/fuzz/FuzzedDataProvider.h`](https://github.com/bitcoin/bitcoin/blob/master/src/test/fuzz/FuzzedDataProvider.h) (added to exceptions)

  On master 5622d8f3156a293e61d0964c33d4b21d8c9fd5e0:
  ```
  $ ./contrib/devtools/copyright_header.py report . | grep zero
    25 with zero copyrights
  ```

  With this PR:
  ```
  $ ./contrib/devtools/copyright_header.py report . | grep zero
     2 with zero copyrights
  ```

  ~I am uncertain about our copyright policy with `build_msvc` and `contrib` directories content, so they are out of scope of this PR.~

ACKs for top commit:
  MarcoFalke:
    ACK fac86ac7b3ceac2f884412c7a9f4bd5bab5e3916

Tree-SHA512: d7832c4a7a1a3b7806119775b40ec35d7982f49ff0e6199b8cee4c0e0a36e68d51728b6ee9924b1c161df4bc6105bd93391b79d42914357fa522f499cb113fa8
2022-04-02 09:19:20 +05:30
Kittywhiskers Van Gogh
8f14c3e2e2 fuzz: add gitlab build support, use clang 2022-03-17 19:23:55 +05:30
fanquake
d9ec960820 Merge bitcoin/bitcoin#22516: macdeploy: alternative info to download the macOS SDK
f8f772dc495f7d3ee02d75d42ee22ceb90c3a693 macdeploy: alternative info to download the macOS SDK (Antoine Poinsot)

Pull request description:

  The previous link wasn't accessible for me, this adds some instructions
  given to me by Hebasto on #bitcoin-core-builds as well as a shasum for
  the archive to quickly check the downloaded one is the right one before
  processing with the entire Guix build.

ACKs for top commit:
  fanquake:
    ACK f8f772dc495f7d3ee02d75d42ee22ceb90c3a693

Tree-SHA512: 620160b593ed8fa4ae4a748b8e72d67b93ff0ec9e6b8ef3c3ac5402c1c48ec0ac325a527b6278cdf84aaf51ba8194d4c366c412ffad141d0412add2710efcff5
2022-03-13 14:52:10 -05:00
Wladimir J. van der Laan
3d6cc69131 Merge #20333: build: remove native_biplist dependency
7087440894a9daa7de806c5aa42d83ad60759c65 depends: native_ds_store 1.3.0 (fanquake)

Pull request description:

  `ds_store` [now takes advantage](36fb607940) of Pythons ability to decode binary [plists](https://docs.python.org/3/library/plistlib.html) (since 3.4), so we can drop its biplist dependency.

  The call to `biplist.Data()` in `custom_dsstore.py` doesn't seem to do anything, and from what I can tell can just be removed. i.e:
  ```diff
  diff --git a/contrib/macdeploy/custom_dsstore.py b/contrib/macdeploy/custom_dsstore.py
  index dc1c1882d..e475bc6c3 100755
  --- a/contrib/macdeploy/custom_dsstore.py
  +++ b/contrib/macdeploy/custom_dsstore.py
  @@ -47,6 +47,7 @@ alias.volume.disk_image_alias.target.filename = package_name_ns + '.temp.dmg'
   alias.volume.disk_image_alias.target.carbon_path = 'Macintosh HD:Users:\x00bitcoinuser:\x00Documents:\x00bitcoin:\x00bitcoin:\x00' + package_name_ns + '.temp.dmg'
   alias.volume.disk_image_alias.target.posix_path = 'Users/bitcoinuser/Documents/bitcoin/bitcoin/' + package_name_ns + '.temp.dmg'
   alias.target.carbon_path = package_name_ns + ':.background:\x00background.tiff'
  +assert(biplist.Data(alias.to_bytes()) == alias.to_bytes())
   icvp['backgroundImageAlias'] = biplist.Data(alias.to_bytes())
   ds['.']['icvp'] = icvp
  ```

ACKs for top commit:
  laanwj:
    ACK 7087440894a9daa7de806c5aa42d83ad60759c65

Tree-SHA512: 8ba3cf561937efe4a3daae8b0cb4de3bf9e425b3a9244161b09d94ee2b1bd4c3e21315fa70e495b19a052aabdc1731b3b6f346b63272d72d2762ced83237d02f
2022-03-05 13:48:08 -06:00