c491368d8cfddf3a5b6d574f10ed67492fcecbed scripts: add MACHO dylib checking to symbol-check.py (fanquake)
76bf97213f4b153dd3ccf1314088a73c4804601d scripts: fix check-symbols & check-security argument passing (fanquake)
Pull request description:
Based on #17857.
This adds dynamic library checks for MACHO executables to symbol-check.py. The script has been modified to function more like `security-check.py`. The error output is now also slightly different. i.e:
```bash
# Linux x86
bitcoin-cli: symbol operator new[](unsigned long) from unsupported version GLIBCXX_3.4
bitcoin-cli: export of symbol vtable for std::basic_ios<char, std::char_traits<char> > not allowed
bitcoin-cli: NEEDED library libstdc++.so.6 is not allowed
bitcoin-cli: failed IMPORTED_SYMBOLS EXPORTED_SYMBOLS LIBRARY_DEPENDENCIES
# RISCV (skips exported symbols checks)
bitcoin-tx: symbol operator new[](unsigned long) from unsupported version GLIBCXX_3.4
bitcoin-tx: NEEDED library libstdc++.so.6 is not allowed
bitcoin-tx: failed IMPORTED_SYMBOLS LIBRARY_DEPENDENCIES
# macOS
Checking macOS dynamic libraries...
libboost_filesystem.dylib is not in ALLOWED_LIBRARIES!
bitcoind: failed DYNAMIC_LIBRARIES
```
Compared to `v0.19.0.1` the macOS allowed dylibs has been slimmed down somewhat:
```diff
src/qt/bitcoin-qt:
/usr/lib/libSystem.B.dylib
-/System/Library/Frameworks/DiskArbitration.framework/Versions/A/DiskArbitration
/System/Library/Frameworks/IOKit.framework/Versions/A/IOKit
/System/Library/Frameworks/Foundation.framework/Versions/C/Foundation
/System/Library/Frameworks/CoreServices.framework/Versions/A/CoreServices
/System/Library/Frameworks/AppKit.framework/Versions/C/AppKit
/System/Library/Frameworks/ApplicationServices.framework/Versions/A/ApplicationServices
/System/Library/Frameworks/CoreFoundation.framework/Versions/A/CoreFoundation
-/System/Library/Frameworks/Security.framework/Versions/A/Security
-/System/Library/Frameworks/SystemConfiguration.framework/Versions/A/SystemConfiguration
/System/Library/Frameworks/CoreGraphics.framework/Versions/A/CoreGraphics
-/System/Library/Frameworks/OpenGL.framework/Versions/A/OpenGL
-/System/Library/Frameworks/AGL.framework/Versions/A/AGL
/System/Library/Frameworks/Carbon.framework/Versions/A/Carbon
/usr/lib/libc++.1.dylib
-/System/Library/Frameworks/CFNetwork.framework/Versions/A/CFNetwork
/System/Library/Frameworks/CoreText.framework/Versions/A/CoreText
/System/Library/Frameworks/ImageIO.framework/Versions/A/ImageIO
/usr/lib/libobjc.A.dylib
```
ACKs for top commit:
laanwj:
ACK c491368d8cfddf3a5b6d574f10ed67492fcecbed
Tree-SHA512: f8624e4964e80b3e0d34e8d3cc33f3107938f3ef7a01c07828f09b902b5ea31a53c50f9be03576e1896ed832cf2c399e03a7943a4f537a1e1c705f3804aed979
2c403279e2f0f7c8c27c56d4e7b0573c59571f0a gitian: Remove codesign_allocate and pagestuff from MacOS build (Andrew Chow)
f55eed251488d70d5e2e3a2965a4f8ec0c476853 gitian: use signapple to create the MacOS code signature (Andrew Chow)
95b06d21852b28712db6c710e420a58bdc1a0944 gitian: use signapple to apply the MacOS code signature (Andrew Chow)
42bb1ea363286b088257cabccb686ef1887c1d3b gitian: install signapple in gitian-osx-signer.yml (Andrew Chow)
Pull request description:
The MacOS code signing issues that were encountered during the 0.21.0 release cycle have shown that it is necessary for us to use a code signing tool for which the source code is available and modifiable by us. Given that there appears to not be such a tool available, I have written such a tool, [signapple](https://github.com/achow101/signapple), that we can use. This tool is able to create a valid MacOS code signature, detach it in a way that we were doing previously, and attach it to the unsigned binary. This tool can also verify that the signature is correct.
This PR implements the usage of that tool in the gitian build for the code signed MacOS binary. The code signer will use this tool to create the detached signature. Gitian builders will use this tool to apply the detached signature. The `gitian-osx-signer.yml` descriptor has been modified to install this tool so that the `detached-sig-apply.sh` script can use it. Additionally, the `codesign_allocate` and `pagestuff` tools are no longer necessary so they are no longer added to the tarball used in code signing. Lastly, both the `detached-sig-create.sh` and `detached-sig-apply.sh` scripts are made to be significantly less complex and to not do unexpected things such as unpacking an already unpacked tarball.
The detached code signature that signapple creates is almost identical to that which we were previously creating. The only difference is that the cpu architecture name is included in the extension (e.g. we have `bitcoin-qt.x86_64sign` instead of `bitcoin-qt.sign`). This was done in order to support signing universal binaries which we may want to do in the future. However signapple can still apply existing code signatures as it will accept the `.sign` extension. If it is desired, it can be modified to produce signatures with just the `.sign` extension. However I do not think it is necessary to maintain compatibility with the old process.
ACKs for top commit:
laanwj:
Code review ACK 2c403279e2f0f7c8c27c56d4e7b0573c59571f0a
Tree-SHA512: 2a0e01e9133f8859b9de26e7e8fe1d2610d2cbdee2845e6008b12c083c7e3622cbb2d9b83c50a269e2c3074ab95914a8225d3cd4108017f58b77a62bf10951e0
7c9e821c4e6cb186208ead9c8df616d1f393a49a scripts: add MACHO NOUNDEFS check to security-check.py (fanquake)
4ca92dc6d3f3e487d63286d8871d1829b3d279ff scripts: add MACHO PIE check to security-check.py (fanquake)
Pull request description:
This uses `otool -vh` to print the mach header and look for the `PIE` flag:
```bash
otool -vh src/bitcoind
Mach header
magic cputype cpusubtype caps filetype ncmds sizeofcmds flags
MH_MAGIC_64 X86_64 ALL LIB64 EXECUTE 24 2544 NOUNDEFS DYLDLINK TWOLEVEL WEAK_DEFINES BINDS_TO_WEAK PIE
```
From [`mach-o/loader.h`](https://opensource.apple.com/source/cctools/cctools-927.0.2/include/mach-o/loader.h.auto.html):
```c
#define MH_PIE 0x200000 /* When this bit is set, the OS will
load the main executable at a
random address. Only used in
MH_EXECUTE filetypes. */
```
ACKs for top commit:
laanwj:
code review ACK 7c9e821c4e6cb186208ead9c8df616d1f393a49a
Tree-SHA512: 5ba2f60440d0e31c70371a355c91ca4f723d80f7287d04e2098bf5b11892cc74216ff8f1454603c4db9675d4f7983614843b992b8dcfca0309aadf2aa7ab2e4b
8f7b93047581c67f2133cdb8c7845471de66c30f Drop the leading 0 from the version number (Andrew Chow)
Pull request description:
Removes the leading 0 from the version number. The minor version, which we had been using as the major version, is now the major version. The revision, which we had been using as the minor version, is now the minor version. The revision number is dropped. The build number is promoted to being part of the version number. This also avoids issues where it was accidentally not included in the version number.
The CLIENT_VERSION remains the same format as previous as previously, as the Major version was 0 so it never actually got included in it.
The user agent string formatter is updated to follow this new versioning.
***
Honestly I'm just tired of all of the people asking for "1.0" that maybe this'll shut them up. Skip the whole 1.0 thing and go straight to version 22.0!
Also, this means that the terminology we commonly use lines up with how the variables are named. So major versions are actually bumping the major version number, etc.
ACKs for top commit:
jnewbery:
Code review ACK 8f7b930475
MarcoFalke:
review ACK 8f7b93047581c67f2133cdb8c7845471de66c30f 🎻
Tree-SHA512: b5c3fae14d4c0a9c0ab3b1db7c949ecc0ac3537646306b13d98dd0efc17c489cdd16d43f0a24aaa28e9c4a92ea360500e05480a335b03f9fb308010cdd93a436
2ecaf214331b506ebfac4f4922241744357d652b gitian: remove execstack workaround for ricv64 & powerpc64le (fanquake)
5baff2b31840bdbc465f55b875aa6e9480288215 build: use focal in gitian descriptors (fanquake)
Pull request description:
This PR changes the gitian descriptors to use Ubuntu Focal (20.04), over Bionic (18.04), moving from GCC 7.5 to GCC 8.4 for native Linux builds, mingw-w64 GCC 7.3 to mingw-w64 GCC 9.3 for Windows builds, while continuing to use GCC 8.4 for all cross builds and Clang 8.0.0 for macOS builds.
It also drops the `-Wl,-z,noexecstack` workaround we've been using for the riscv64 and powerpc64le hosts, as it's no-longer needed. One new package is installed in the osx build, `libtinfo5`, as libtinfo5.so is required by our downloaded Clang 8.
A bump to Focal will at least be required if we want to update to a newer Qt (5.15, #19716) for 22.0, as we need a newer version of [`g++-mingw-w64`](https://packages.ubuntu.com/focal/g++-mingw-w64-x86-64) and the [`mingw-w64`](https://mingw-w64.org/doku.php) headers. This can still be done while continuing to use GCC 8.4 for Linux builds (see below), however the newer `g++-mingw-w64` will be based off of GCC 9.3.
**Some considerations**
GCC 9 is affected by #20005 "memcmp with constants that contain zero bytes are broken in GCC", and the newer `g++-mingw-w64` will be based off of GCC 9.3.
The `--no-*` variants of the Windows linker flags (i.e `--no-dynamicbase`) we use to [test our `security-check.py` script](16b784d953/contrib/devtools/test-security-check.py (L53)) are not patched into the mingw binutils in Focal (they have been re-added in Groovy (20.10)). This isn't currently an issue, however, we might add a call to `test-security-check` for Guix (#20980), and if we wanted to do the same for gitian, it would not work. Note how it's quite "easy" for us to apply the `--no-*` variant patch to our Guix build; it would be quite a bit harder to do in Gitian.
Gitian Builds @ 2ecaf214331b506ebfac4f4922241744357d652b
#### Linux
```bash
8882ea78486fbae4fac574b9089eb1107c6372d0dd7dfcda4f0f930576f9d6c1 bitcoin-2ecaf214331b-aarch64-linux-gnu-debug.tar.gz
50a9e30943b4eee5163edff3331241e745ff32a2c4463c21a6fdc5986e2d0383 bitcoin-2ecaf214331b-aarch64-linux-gnu.tar.gz
ec4e55a447fddf033fee33cd5f22bfeda3c3612f059194bcf6238859f7989d7a bitcoin-2ecaf214331b-arm-linux-gnueabihf-debug.tar.gz
444fe1b3b933c00bcbd4a9d86888cff3b61c1215b1debccd2843e842d1224777 bitcoin-2ecaf214331b-arm-linux-gnueabihf.tar.gz
88e486ff465980dc1a4aab9687d142ec6f727ed2c52cf539f69db2877dee83b2 bitcoin-2ecaf214331b-powerpc64-linux-gnu-debug.tar.gz
66144ac264c65cada9d86446e6026c85b04fb88198b8f41b42840f6031db3e6c bitcoin-2ecaf214331b-powerpc64-linux-gnu.tar.gz
34bcc13d78d929d575e34e77a6672f23ca7ea23230b28ec2eed563889352ba86 bitcoin-2ecaf214331b-powerpc64le-linux-gnu-debug.tar.gz
b4c5f959664f3063df4330edfe343c17120eb6b556ee1c15c4aeb2c1c54ffd49 bitcoin-2ecaf214331b-powerpc64le-linux-gnu.tar.gz
918fa72ab6f6ebce4e9663c93f72fe26651c260477cbb54749f7eb61438b5cc1 bitcoin-2ecaf214331b-riscv64-linux-gnu-debug.tar.gz
f704f9f8c053ffe37d854e2e81e0f4c0614c435dad7f5d82518c681b73a76ae6 bitcoin-2ecaf214331b-riscv64-linux-gnu.tar.gz
b59e3a62f1df9d79f30e916b3c9655f654036fe3a420040c53acc8dd9f4162c5 bitcoin-2ecaf214331b-x86_64-linux-gnu-debug.tar.gz
a4dc9ca877cc97544e65db11be38406d16f15d74fcdcd2318bb92474729bc60d bitcoin-2ecaf214331b-x86_64-linux-gnu.tar.gz
b40ba2d5da498330ade92a4ccebcceb1452b94c8ffeacb336f87e93b5c88d8af src/bitcoin-2ecaf214331b.tar.gz
af6ebc91147778e4e6705eade62608dde4d6e60522d79087fa9129bdb7c01199 bitcoin-core-linux-22-res.yml
```
#### Windows
```bash
121a3970a6911cb8c453b2ce37d03f6cbb43333e29db8fa516c68563fb367f43 bitcoin-2ecaf214331b-win-unsigned.tar.gz
6294e9efebe935092f9ba119dc60ad4094f18b51c4181324e54d3057524d6101 bitcoin-2ecaf214331b-win64-debug.zip
5b5a236b63e67f5f6c07ad9aa716aa7b72fb63722c96798b332c6d164738f9cf bitcoin-2ecaf214331b-win64-setup-unsigned.exe
c1fa5894c5e02a201637567c80b9bde9024f44673dcd06fd4d489c1709179279 bitcoin-2ecaf214331b-win64.zip
b40ba2d5da498330ade92a4ccebcceb1452b94c8ffeacb336f87e93b5c88d8af src/bitcoin-2ecaf214331b.tar.gz
665fd7eb61aed368150db58a254f15fb5efb51a4efa5abcc52571cb7a1a5de22 bitcoin-core-win-22-res.yml
```
#### macOS
```bash
6a1deae7662aa782baa82a42590f862c6bcdc4f4e38daa9b8c2a9eed1fbb5397 bitcoin-2ecaf214331b-osx-unsigned.dmg
1ee843266e84928a4323fa255c833528c2617a2c9fd2f98fb26ba19bbfc1227b bitcoin-2ecaf214331b-osx-unsigned.tar.gz
097b64dadc167d8e5b733421bf1541a40760ad952990f7cf3f35adc6ae2616d0 bitcoin-2ecaf214331b-osx64.tar.gz
b40ba2d5da498330ade92a4ccebcceb1452b94c8ffeacb336f87e93b5c88d8af src/bitcoin-2ecaf214331b.tar.gz
6e378fb543928e40c7119b96be6ff773d38506a9a888f8b02c7f1b8a0801a80e bitcoin-core-osx-22-res.yml
```
ACKs for top commit:
laanwj:
Build script changes review ACK 2ecaf214331b506ebfac4f4922241744357d652b
Tree-SHA512: 975d5830b787d2e08988f43cbc6e839294171c1d94c8219636308b05f9b77041421612ae67be24a631674670cfc9c2d96d8177f2b3158a78fc3deea19631febf
faa2f06f5eaf8578873495f44603ee74d7a1abf4 scripted-diff: [build] Ensure source tarball has leading directory name (MarcoFalke)
Pull request description:
This has been fixed in 0.20, so it needs to be fixed on master as well to avoid a regression
#18945
ACKs for top commit:
laanwj:
ACK faa2f06f5eaf8578873495f44603ee74d7a1abf4
hebasto:
ACK faa2f06f5eaf8578873495f44603ee74d7a1abf4, tested gitian builds only.
promag:
ACK faa2f06f5eaf8578873495f44603ee74d7a1abf4.
Tree-SHA512: e3b025c29c45b025002abc35262bb5d771f6cbd807f1c256c477c243685e93cd43ad9f642b38e3cf218590912abe6ea0ddfec3bfbef36f99080aad74ed6cc0af
bfe1ba2f5b36056e0c41edf8206b93d3d83098df rel-builds: Specify core.abbrev for git-rev-parse (Carl Dong)
27e63e01cce368d67092de8f0c736927d6f6aa69 build: Accomodate makensis v2.x (Carl Dong)
1f2c39a30e0f82046c7aecddfda3eb99cb536816 guix: Remove logical cores requirement (Carl Dong)
a4f6ffa71e335d4b2a6bf525b7f416968f9cd9f7 lint: Also enable source statements for non-gitian (Carl Dong)
d256f91cb1b0d6ff5170106b99b0266cbe51f5a2 rel-builds: Directly deploy win installer to OUTDIR (Carl Dong)
fa791da02f9684e3fd554b687fb692ae6a23d65a nsis: Specify OutFile path only once (Carl Dong)
14701604d0904bc5bbf1c67de08f8ee6d3215523 guix: Expose GIT_COMMON_DIR in container as readonly (Carl Dong)
f5a6ac4f48b18f93050d77bcb23f9cf45ec34647 guix: Make source tarball using git-archive (Carl Dong)
395c1137f630dc495ffb2752a23bc1dfd470ee53 gitian: Limit sourced script to just assignments (Carl Dong)
Pull request description:
Based on: #18556
Related: https://github.com/bitcoin/bitcoin/pull/17595#discussion_r399728721
ACKs for top commit:
fanquake:
ACK bfe1ba2f5b36056e0c41edf8206b93d3d83098df - I agree with Carl, and am going to merge this. I'd like for Linux Guix builds to be working again, and we can rebase #18818.
Tree-SHA512: c87ada7e3de17ca0b692a91029b86573442ded5780fc081c214773f6b374a0cdbeaf6f6898c36669c2e247ee32aa7f82defb1180f8decac52c65f0c140f18674
2aa48edec0101f8a77a2189244fc62722ff7a123 refactor: Drop unused ${WRAP_DIR}/${HOST} directory (Hennadii Stepanov)
1362be044724bb49d785ca2e296a3b43343c1690 build: Drop make dist in gitian builds (Hennadii Stepanov)
Pull request description:
After the merge of #18331, the packaged source tarball is created by `git archive`, but the binaries are built from another one which is made by `make dist`.
With this PR the only source tarball, created by `git archive`, is used both for binaries building and for packaging to users.
Close#16588.
Close#18547.
As a good side-effect, #18349 becomes redundant.
**Change in behavior**
The following variables 1b151e3ffc/configure.ac (L2-L6)
are no longer used for naming of directories and tarballs.
Instead of them the gitian descriptors use a git tag (if available) or a commit hash.
---
Also a small refactor commit picked from #18404.
ACKs for top commit:
dongcarl:
ACK 2aa48edec0101f8a77a2189244fc62722ff7a123
MarcoFalke:
ACK 2aa48edec0101f8a77a2189244fc62722ff7a123
fanquake:
ACK 2aa48edec0101f8a77a2189244fc62722ff7a123 - I've had a quick look over this, and don't want to block merging if this actually gets as closer to finally having this all sorted out. Obviously we've still got #18741, and after speaking to Carl this morning, there will likely be even more changes after that (not Guix specific).
Tree-SHA512: d3b16f87e48d1790a3264940c28acd5d881bfd10f3ce94fb0c8a6af76d8039289d01e0cd4972adac49ae24362857251f6c1e5e09e3e9fbf636c10708b4015a7c
e4d366788bc2e8dce8e6ca572fce08d913d15d6b build: Drop needless EXTRA_DIST content (Hennadii Stepanov)
6c4da59f5b5b3c40526d38965d4ffa7fd59f2ebc build: Drop SOURCEDIST reordering (Hennadii Stepanov)
5e6b8b391243016cb06e9e107c2e6a13a744b31e build: Use git archive as source tarball (Hennadii Stepanov)
Pull request description:
This PR:
- is an alternative to #17104
- closes#16734
- closes#6753
The idea is clear described by some developers:
- [MarcoFalke](https://github.com/bitcoin/bitcoin/pull/17097#issuecomment-540691850):
> This whole concept of explicitly listing each and every file manually (or with a fragile wildcard) is an obvious sisyphean task. I'd say all we need to do is run git archive and be done with it forever, see #16734, #6753, #11530 ...
- [laanwj](https://github.com/bitcoin/bitcoin/pull/17097#issuecomment-540706025):
> I agree, I've never been a fan of it. I don't think we have any files in the git repository we don't want to ship in the source tarball.
---
The suggested changes have a downside which is pointed by [**luke-jr**](https://github.com/bitcoin/bitcoin/pull/17104#issuecomment-540828045):
> ... but the distfile needs to include autogen-generated files.
This means that a user is not able to run `./configure && make` right away. One must run `./autogen.sh` at first.
Here are opinions about mandatory use of `./autogen.sh`:
- [ryanofsky](https://github.com/bitcoin/bitcoin/issues/16734#issuecomment-534139356):
> It's probably ok to require autogen. I think historically configure scripts were supposed to work on obscure unix systems that would just have a generic shell + make tool + c compiler, and not necessarily need gnu packages like m4 which are needed for autogen.
- [laanwj](https://github.com/bitcoin/bitcoin/issues/16734#issuecomment-540729483):
> I also think it's fine to require autogen. What is one dependency more, if you're building from source.
---
~Also this PR provides Windows users with ZIP archives of the sources. Additionally the commit ID is stored in these ZIP files as a file comment:~
---
Note for reviewers: please verify is `git archive` output deterministic?
ACKs for top commit:
MarcoFalke:
re-ACK e4d366788bc2e8dce8e6ca572fce08d913d15d6b, only change is adding two dots in a the path 🛳
laanwj:
ACK e4d366788bc2e8dce8e6ca572fce08d913d15d6b
Tree-SHA512: d1153d3ca4a580696019b92be3555ab004d197d9a2146aacff9d3150eb7093b7d40eebd6eea12d861d93ff62d62b68706e04e64dbe5ea796ff6757486e462193
fae75306bac4c82dd07a1b85ce5dfb020e052fe8 scripted-diff: Set gitian arch back to amd64 (MarcoFalke)
Pull request description:
This was required to allow gitian builds on non-amd64 architecture, however, it seems to break the current builds (with lxc), see https://github.com/bitcoin/bitcoin/pull/17409#issuecomment-554099626
Also, the gititan builds wouldn't be deterministic across arches anyway, see #17468
So instead of wasting more time on this, revert the change and hope that guix allows to compile on non-amd64 architectures.
Top commit has no ACKs.
Tree-SHA512: 801e9a30ae1b0882ef45d5eb3a3cf80f3ace3b99db046069dbd95b6162119e977e3cf3134287d1ac5d09483906206acc71e1ac34d6b74dbc533d46aaf73f5cc2
333362991c5219e6d9ede5fa54328bcbb0277cf2 doc: Explain $LIB in LD_PRELOAD in gitian descriptors (MarcoFalke)
fab9850ef4a76e4f09278909a2865e5175661653 scripted-diff: Avoid hardcoded libfaketime dir in gitian (MarcoFalke)
Pull request description:
Without this gitian prints warnings for me:
```
ERROR: ld.so: object '/usr/lib/x86_64-linux-gnu/faketime/libfaketime.so.1' from LD_PRELOAD cannot be preloaded (cannot open shared object file): ignored.
```
```
$ ls /usr/lib/aarch64-linux-gnu/faketime/libfaketime.so.1
/usr/lib/aarch64-linux-gnu/faketime/libfaketime.so.1
```
ACKs for top commit:
laanwj:
ACK 333362991c5219e6d9ede5fa54328bcbb0277cf2
Tree-SHA512: 3e7c4471b69c2ae38c29d0cc0db8b9eae0912085299d7f5ac67eeb4b6a2fdc7eb23d806eeeae0b0c2da22d6d1ba82513cab23652876b97aada9928b2c7d38e7e
993aa414d3 scripted-diff: gitian: Use REFERENCE_DATETIME directly. (Carl Dong)
Pull request description:
Fixes regression introduced by #16141.
```
-BEGIN VERIFY SCRIPT-
sed -i 's#\$REFERENCE_DATE\\\\\\ \$REFERENCE_TIME#\$REFERENCE_DATETIME#g' contrib/gitian-descriptors/*
-END VERIFY SCRIPT-
```
-----
Note that this could have been fixed by escaping properly, but using `REFERENCE_DATETIME` directly is simpler.
Future note: `REFERENCE_{DATE{,DATETIME},TIME}` is a bit ridiculous. At the very _least_ gitian should use epoch, as it is the most parse-able, and preferably set SOURCE_DATE_EPOCH.
ACKs for commit 993aa4:
Tree-SHA512: 8457e5fffde66e1d2b846547b6807416b884c171f63569f76dfefd498d2a58ad6f9eb93931eb6cfc7ff38c6b460b0c488ca87d1a68bc630c48f365a74b6ee163
* lint: Skip shell linting if gawk is not installed
* lint: Skip Gitian descriptor scripts checking if jq is not installed
* ci: Install gawk and jq
`yq` requires `jq`
* Fix shellcheck warnings
b8705a091565d4373e0e8ad4aa764cfd906708db Set 'distro' explicitly (Hennadii Stepanov)
Pull request description:
The [gitian-builder](https://github.com/devrandom/gitian-builder) implicitly uses `ubuntu` as a default distro.
[bin/gbuild#L237](81edd2fc8e/bin/gbuild (L237)):
```ruby
distro = build_desc["distro"] || "ubuntu"
```
This PR sets a gitian building distro explicitly in description files.
Tree-SHA512: d2a692047f3466a5c637433610854d1100fe68a78fb03b4a81c70911fa14b0228d9cf25bcf115930aab9cc8c5063dacaf79bcd151f1c3f6fee6419389aefcb8b
32da92bdf6bb55d6d312b0f85797d439cc942db5 gitian: Improve error handling (Wladimir J. van der Laan)
Pull request description:
Improve error handling in gitian builds:
- Set fail-on-error and pipefail flag, this causes a command to fail when either of the pipe stages fails, not only when the last of the stages fails, so this improves error detection.
- Also use `xargs` instead of `find -exec`, because `find` will not propagate errors in the executed command, but `xargs` will.
This will avoid some issues like #15541 where non-determinism is silently introduced due to errors caused by environment conditions (such as lack of disk space in that case).
Tree-SHA512: d5d3f22ce2d04a75e5c25e935744327c3adc704c2d303133f2918113573a564dff3d3243d5569a2b93ee7eb0e97f8e1b1ba81767e966af9015ea711a14091035
* build: Add cmake as depends package (cmake.mk)
The bls-signatures library requires cmake 3.14
* depends: Update chia_bls to version 1.0.0 of dashpay/bls-signatures
* depends: Rename package chia_bls to bls-dash
* depends: Disable blspy/tests/benchmarks build for chia_bls
Note: Building with tests would require the following:
depends: Fix macOS build for versions < 10.12
Seems like older versions of macOS pretend to support c++17 std libs but do not have/have issues with uncaught_exceptions. "Catch", the testing framework used in the bls lib wants those by default but setting `DCATCH_CONFIG_NO_CPP17_UNCAUGHT_EXCEPTIONS` disables them which should be just fine to do in all cases here.
---
depends/packages/chia_bls.mk | 2 ++
1 file changed, 2 insertions(+)
diff --git a/depends/packages/chia_bls.mk b/depends/packages/chia_bls.mk
index e79a85844f..25749b74ab 100644
--- a/depends/packages/chia_bls.mk
+++ b/depends/packages/chia_bls.mk
@@ -28,6 +28,8 @@ define $(package)_set_vars
$(package)_config_opts_darwin+= -DCMAKE_AR="$(host_prefix)/native/bin/$($(package)_ar)"
$(package)_config_opts_darwin+= -DCMAKE_RANLIB="$(host_prefix)/native/bin/$($(package)_ranlib)"
endif
+
+ $(package)_cppflags+= -DCATCH_CONFIG_NO_CPP17_UNCAUGHT_EXCEPTIONS
endef
define $(package)_config_cmds
--
* depends: Drop obsolete variable
* bls: Integrate the upgraded version into the codebase
* depends: Pre-fetch relic to fix gitian
Can fetch apt packages only when building with lxc
Co-authored-by: UdjinM6 <UdjinM6@users.noreply.github.com>
000000035b20402dea3e8168165cd4eefdc97539 Obsolete #!/bin/bash shebang (DesWurstes)
Pull request description:
> `#!/bin/bash` assumes it is always installed to `/bin/` which can cause issues
> `#!/usr/bin/env bash` searches the user's `PATH` to find the `bash` binary
Details: https://github.com/dylanaraps/pure-bash-bible#obsolete-syntax
I'm open to comments: Should I also fix `#!/bin/sh`?
Tree-SHA512: b47bb4828116aa119f1899c68fee081270d51a898535490b9c616bf0f3660ad953f29c361eafc759bc64cdd54ee6eeecb2d79e9fdb5291a996a515c719805476
3828a79711 scripted-diff: prefer MAC_OSX over __APPLE__ (fanquake)
fa6e841e89 gui: remove macOS ProgressBar workaround (fanquake)
68c272527f gui: remove SubstituteFonts (fanquake)
6c6dbd8af5 doc: mention that macOS 10.10 is now required (fanquake)
84b0cfa8b6 release: bump minimum required macOS to 10.10 (fanquake)
26b15df99d depends: set OSX_MIN_VERSION to 10.10 (fanquake)
Pull request description:
Closes#13362
d99abfddb0c8f2111340a6127e77cc686e0043d8
This workaround should no longer be required, as it should have only been in use when compiled with the 10.7 SDK, which we haven't been building with for a while now.
5bc5ae30982a0f0f6a9804b05d99434af770c724
The bugreport linked with this code is for an unrelated? issue, however from what I can tell the correct QTBUG is this one https://bugreports.qt.io/browse/QTBUG-20880. Reading though the discussion there, it seems that the way progress bars are animated changed in macOS 10.10.
Qt was patched [here (5.5+)](https://codereview.qt-project.org/#/c/112379/):
> Disable progress bar animations on 10.10 Yosemite and higher - the native style does not animate them any more. Keep the indeterminate progress bar animation.
Given all of that, I don't think this is worth keeping around, as it would seem to only be useful in the case that a macOS user is compiling with a Qt < 5.5. That should be pretty unlikely, as we don't support downloaded Qt binaries, and brew currently provides [5.11.1](571b46213c/Formula/qt.rb).
Tree-SHA512: 4278cb30cc9bcb313e166129ecf032c808995f8b51a3123637c47860a0010ac88f86f82ec44792153b6b1e5cca595f25013b2eaeae80194647b9ce4f7eaf32c1
4452829 gitian: quick hack to fix version string in releases (Cory Fields)
Pull request description:
Credit: @luke-jr
Release version strings were broken in Gitian by #7522. This is a minimal fix suitable for 0.15.
After this, we should fix up version handling for good so that gitian packages the correct string in the release tarball, so that git is not required to get the tag name.
Tree-SHA512: fa609a744c46306b0809f08fed6e96eff41b13e82f3e213711e4abef370558b64a68972f283a038330882cb6c40b32547fbb0f89b8058cc2c6025bff134473c3
* Remove use of -rdynamic
This causes check-symbols to fail horribly and also turned out to be not
required when using libbacktrace. It was only required when using
"backtrace()" from "<execinfo.h>"
* Remove spurious ], from configure.ac
* Add -DENABLE_STACKTRACES=1 to CMakeLists.txt
* Remove unused method my_backtrace_simple_callback
* Use fs::path().filename() instead of basename()
* Add static g_exeFileName and g_exeFileBaseName
* Use .exe.dbg file when available
* Use uint64_t instead of uintptr_t
* Implement GetBaseAddress() for unix and win32
* Implement unified crash_info and use it everywhere before printing crash info
* Print a serialized version of crash_info when there is no debug info
* Implement "-printcrashinfo" command line option
* Compile stacktrace support unconditionally and only make crash hooks conditional
This also renames the --enable-stacktraces option to --enable-crash-hooks
* Enable crash hooks in win/linux Gitian builds
* Try to load .debug file on MacOS and enable crash hooks for osx Gitian builds
* Check for dsymutil and if it needs --flat
* Create .debug files in osx Gitian build
* Handle review comments
* Also print crash description when no stacktrace is available
* Unconditionally add -g1 debug information
Instead of making it dependent on "--enable-crash-hooks". We will need the
debug info every time now, even in release builds.
* Put MacOS debug info into dSYM symbols instead of plain .debug files
* Implement MacOS specific GetBaseAddress
ed1fcdc Bugfix: Detect genbuild.sh in repo correctly (Luke Dashjr)
e98e3dd Bugfix: Only use git for build info if the repository is actually the right one (Luke Dashjr)
Tree-SHA512: 510d7ec8cfeff4e8e0c7ac53631eb32c7acaada7017e7f8cc2e6f60d86afe1cd131870582e01022f961c85a783a130bcb8fef971f8b110070c9c02afda020726
The cache can become quite large and uncompressing/recompressing it is slow.
Better to let ccache handle compression as it will then only perform it
on files which are actually used.
* Add stdin, stdout, stderr to ignored export list
* Merge #13171: Change gitian-descriptors to use bionic instead
c1afe3232fa10e290fb355cf37ea4c7bc1084065 Change gitian-descriptors to use bionic instead (Chun Kuan Lee)
Pull request description:
I have tested this with Ubuntu Bionic host with lxc 3.0.0, the gitian-build for Windows and MacOSX work fine, but there is an issue about it for Linux. Failed at check-symbol:
```
test/test_bitcoin: symbol __divmoddi4 from unsupported version GCC_7.0.0
test/test_bitcoin: symbol log2f from unsupported version GLIBC_2.27
qt/bitcoin-qt: symbol __divmoddi4 from unsupported version GCC_7.0.0
qt/bitcoin-qt: symbol log2f from unsupported version GLIBC_2.27
```
I think this should be fixed in `./configure --enable-glibc-back-compat`
Should not be merged before #13177devrandom/gitian-builder#178Close#12511
Tree-SHA512: 257d75d5b6864e105279f7a7b992fbbd7903cdbe3300b66dacec0a783d267707d9dbbfe0e64a36983ca1eca50a2a5e1cdb222b6d3745ccc3e5fc5636c88b581f
* Use IN6ADDR_ANY_INIT instead of in6addr_any
This is the same fix as fc6a9f2ab1
Couldn't backport the original commit as we are missing some refactorings.
* Use subdirectory for depends cache in gitian builds
* Make timestamps of wrappers deterministic
* Use ccache in gitian builds
* Upgrade ccache to latest version (3.4.2)
* Build the branch that belongs to the Jenkins build instead of develop
We mostly switched to python3, so we should also install it in gitian
builds. Especially the osx build needs as it otherwise fails due to
missing setuptools.
