647f7e5f1da1089d451f3c431efc635b8e87b064 guix: Also sort SHA256SUMS.part (Carl Dong)
dc4137a60c99979b89f75d2bddba96d043f387b8 guix: Build depends/qt with our platform definition (Carl Dong)
16b0a936e15b81710755303e11ef51f608b61475 guix: Rebase toolchain on glibc 2.24 (2.27 for riscv64) (Carl Dong)
Pull request description:
After this PR, we'll have the following:
- riscv64 -> build with a toolchain targeting glibc 2.27
- everything else -> builds with a toolchain targeting glibc 2.24, but will not have symbols > 2.17 (checked by `symbol-check.py`)
ACKs for top commit:
achow101:
reACK 647f7e5f1da1089d451f3c431efc635b8e87b064
hebasto:
ACK 647f7e5f1da1089d451f3c431efc635b8e87b064
MarcoFalke:
review ACK 647f7e5f1da1089d451f3c431efc635b8e87b064
fanquake:
ACK 647f7e5f1da1089d451f3c431efc635b8e87b064 - documentation can be fixed shortly.
Tree-SHA512: ddff57a5d7c053687b0a273720d4ad7d28c6fc8816226d4304869284d017af5e3630d4b57565d91e74f2e1b7583c9c83ee8b2e5e70e41d619ab618e602c97a94
e2c40a4ed5272d72fea997bd936fba28bb753226 guix-attest: Error out if SHA256SUMS is unexpected (Carl Dong)
4cc35daed557f38b080360a89036b2e97a6f78c2 Rewrite guix-{attest,verify} for new hier (Carl Dong)
28a9c9b83924f585b397f0f3b8e9e73780ac0ad6 Make SHA256SUMS fragment right after build (Carl Dong)
Pull request description:
Based on: #22075
Code reviewers: I recommend reading the new `guix-{attest,verify}` files instead of trying to read the diff
The following changes resolve many usability improvements which were pointed out to me:
1. Some maintainers like to extract their "uncodesigned tarball" inside the `output/` directory, resulting in the older `guix-attest` mistakenly attesting to the extracted contents
2. Maintainers whose GPG keys reside on an external smartcard often need to physically interact with the smartcard as a way to approve the signing operation, having one signature per platform means a lot of fidgeting
3. Maintainers wishing to sign on a separate machine now has the option of transferring only a subtree of `output/`, namely `output/*/SHA256SUMS.part`, in order to perform a signature (you may need to specify an `$OUTDIR_BASE` env var)
4. An `all.SHA256SUMS` file should be usable as the base `SHA256SUMS` in bitcoin core torrents and on the release server.
For those who sign on an separate machine than the one you do builds on, the following steps will work:
1. `env GUIX_SIGS_REPO=/home/achow101/guix.sigs SIGNER=achow101 NO_SIGN=1 ./contrib/guix/guix-attest`
2. Copy `/home/achow101/guix.sigs/<tag>/achow101` (which does not yet have signatures) to signing machine
3. Sign the `SHA256SUMS` files:
```bash
for i in "<path-to-achow101>/*.SHA256SUMS"; do
gpg --detach-sign --local-user "<your-key-here>" --armor --output "$i"{.asc,}
done
```
5. Upload `<path-to-achow101>` (now with signatures) to `guix.sigs`
-----
After this change, output directories will now include a `SHA256SUMS.part` fragment, created immediately after a successful build:
```
output
└── x86_64-w64-mingw32
├── bitcoin-4e069f7589da-win64-debug.zip
├── bitcoin-4e069f7589da-win64-setup-unsigned.exe
├── bitcoin-4e069f7589da-win64.zip
├── bitcoin-4e069f7589da-win-unsigned.tar.gz
└── SHA256SUMS.part
```
These `SHA256SUMS.part` fragments look something like:
```
3ebd7262b1a0a5bb757fef1f70e7e14033c70f98c059bc4dbfee5d1992b25825 dist-archive/bitcoin-4e069f7589da.tar.gz
def2e7d3de5ab3e3f955344e75151df4f33713f9101f5295bd13c9375bdf633b x86_64-w64-mingw32/bitcoin-4e069f7589da-win64-debug.zip
643049fe3ee4a4e83a1739607e67b11b7c9b1a66208a6f35a9ff634ba795500e x86_64-w64-mingw32/bitcoin-4e069f7589da-win64-setup-unsigned.exe
a247a1ccec0ccc2e138c648284bd01f6a761f2d8d6d07d91b5b4a6670ec3f288 x86_64-w64-mingw32/bitcoin-4e069f7589da-win-unsigned.tar.gz
fab76a836dcc592e39c04fd2396696633fb6eb56e39ecbf6c909bd173ed4280c x86_64-w64-mingw32/bitcoin-4e069f7589da-win64.zip
```
Meaning that they are valid `SHA256SUMS` files when `sha256sum --check`'d at the `guix-build-*/output` directory level
When `guix-attest` is invoked, these `SHA256SUMS.part` files are combined and sorted (by `-k2`, `LC_ALL=C`) to create:
1. `noncodesigned.SHA256SUMS` for a manifest of all non-codesigned outputs, and
3. `all.SHA256SUMS` for a manifest of all outputs including non-codesigned outputs
Then both files are signed, resulting in the following `guix.sigs` hierarchy:
```
4e069f7589da/
└── dongcarl
├── all.SHA256SUMS
├── all.SHA256SUMS.asc
├── noncodesigned.SHA256SUMS
└── noncodesigned.SHA256SUMS.asc
```
ACKs for top commit:
achow101:
ACK e2c40a4ed5272d72fea997bd936fba28bb753226
hebasto:
ACK e2c40a4ed5272d72fea997bd936fba28bb753226, tested on Linux Mint 20.1 (x86_64) with and w/o `NO_SIGN=1`. Changes in `contrib/guix/libexec/codesign.sh` and `contrib/guix/guix-verify` are reviewed only.
Tree-SHA512: 618aacefb0eb6595735a9ab6a98ea6598fce65f9ccf33fa1e7ef93bf140c0f6cfc16e34870c6aa3e4777dd3f004b92a82a994141879870141742df948ec59c1f
108a6be92adc1e80839d90b552e72b8142140f6c guix: Check for disk space availability before building (Carl Dong)
d7dec89091ee4a456ff64ad7ce675ae6813668f1 guix: Remove dest if OUTDIR mv fails (Carl Dong)
Pull request description:
There seems to be some corner cases that can be hit when guix scripts unexpectedly fail in the middle of operation, see: https://gnusha.org/bitcoin-builds/2021-05-24.log
- Perform an early disk space check for `guix-build`
- Overwrite existing output directory after a successful build (the existing one might be malformed), and cleanup output directory if the `mv` somehow fails
ACKs for top commit:
laanwj:
Tested ACK 108a6be92adc1e80839d90b552e72b8142140f6c
achow101:
ACK 108a6be92adc1e80839d90b552e72b8142140f6c
Tree-SHA512: cf6438317da40bf55714cd2d8cce859b3d435cc66cabefe8d4a53552d7880966acfe84ffe8fadf1c80e368ae6b037992258a6d409df85ffc6ce8bf780e98e2e5
ee883201cf134952284632e9e9ae72bf1c8c792f guix: repro: Sort find output in libtool for gcc-8 (Carl Dong)
ee0a67c32a8861eab650bf8894af06807578eba0 codesigning: Use SHA256 as digest for osslsigncode (Windows) (Carl Dong)
38eb91eb0616ed6dbe34c23e11588d130fef07f8 guix: Add codesigning functionality (Carl Dong)
bac2690e6f683fcedb883fe1d32f3c33c628a141 guix: Package codesigning tools (Carl Dong)
0a2176d47767972e4cd5ed302c1dbeedece1708b guix: Reindent existing manifest.scm (Carl Dong)
c090a3e9238ba2df07875b4708e908d8dca4ed9b Makefile.am: use APP_DIST_DIR instead of hard-coding dist (Carl Dong)
Pull request description:
This is the last PR before we reach feature-parity with the Gitian process!
Note: I tried using the `Makefile` inside the distsrc to make the dmg instead of manually listing out the commands, but `make` seems to want to re-make a lot of other files which broke the dmg.
The workflow looks something like this:
1. `env [ FOO=bar... ] ./contrib/guix/guix-build` (add additional env vars as necessary)
2. Codesigners only:
1. Copy `guix-build-<short-id>/output/x86_64-apple-darwin18/bitcoin-<short-id>-osx-unsigned.tar.gz` and `guix-build-<short-id>/output/x86_64-w64-mingw32/bitcoin-<short-id>-win-unsigned.tar.gz` to signing computer
2. Codesign with `./detached-sig-create.sh` inside the tarball
3. Upload contents of `signature-{osx,win}.tar.gz` to https://github.com/bitcoin-core/bitcoin-detached-sigs (as a new tag)
3. Checkout new tag for `bitcoin-core/bitcoin-detached-sigs` with the detached signatures
4. `env [ FOO=bar... ] DETACHED_SIGS_REPO=<path/to/bitcoin-detached-sigs> ./contrib/guix/guix-codesign` (modify env vars as necessary)
5. Make sure `guix.sigs` is cloned and updated
6. `env GUIX_SIGS_REPO=<path/to/guix.sigs> SIGNER=0x96AB007F1A7ED999=dongcarl ./contrib/guix/guix-attest` (modify env vars as necessary)
7. Commit your new signatures and SHA256SUMS in `guix.sigs`
8. Optionally, after there are multiple signatures in `guix.sigs`: `env GUIX_SIGS_REPO=<path/to/guix.sigs> ./contrib/guix/guix-verify`
ACKs for top commit:
laanwj:
Tested ACK ee883201cf134952284632e9e9ae72bf1c8c792f
achow101:
ACK ee883201cf134952284632e9e9ae72bf1c8c792f
Tree-SHA512: e812a07a5f19f900600c70cb9c717769ef544a6c0c12760b5558b76b6b37df863257f3dbf38b0757e6e06e334470267e94c9f2bdbc27409d6837b1a0bfc6acbc
