mirror of
https://github.com/dashpay/dash.git
synced 2024-12-27 04:52:59 +01:00
56d2bc249b
24 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
fanquake
|
56d2bc249b |
Merge bitcoin/bitcoin#22182: guix: Overhaul how guix-{attest,verify} works and hierarchy
e2c40a4ed5272d72fea997bd936fba28bb753226 guix-attest: Error out if SHA256SUMS is unexpected (Carl Dong)
4cc35daed557f38b080360a89036b2e97a6f78c2 Rewrite guix-{attest,verify} for new hier (Carl Dong)
28a9c9b83924f585b397f0f3b8e9e73780ac0ad6 Make SHA256SUMS fragment right after build (Carl Dong)
Pull request description:
Based on: #22075
Code reviewers: I recommend reading the new `guix-{attest,verify}` files instead of trying to read the diff
The following changes resolve many usability improvements which were pointed out to me:
1. Some maintainers like to extract their "uncodesigned tarball" inside the `output/` directory, resulting in the older `guix-attest` mistakenly attesting to the extracted contents
2. Maintainers whose GPG keys reside on an external smartcard often need to physically interact with the smartcard as a way to approve the signing operation, having one signature per platform means a lot of fidgeting
3. Maintainers wishing to sign on a separate machine now has the option of transferring only a subtree of `output/`, namely `output/*/SHA256SUMS.part`, in order to perform a signature (you may need to specify an `$OUTDIR_BASE` env var)
4. An `all.SHA256SUMS` file should be usable as the base `SHA256SUMS` in bitcoin core torrents and on the release server.
For those who sign on an separate machine than the one you do builds on, the following steps will work:
1. `env GUIX_SIGS_REPO=/home/achow101/guix.sigs SIGNER=achow101 NO_SIGN=1 ./contrib/guix/guix-attest`
2. Copy `/home/achow101/guix.sigs/<tag>/achow101` (which does not yet have signatures) to signing machine
3. Sign the `SHA256SUMS` files:
```bash
for i in "<path-to-achow101>/*.SHA256SUMS"; do
gpg --detach-sign --local-user "<your-key-here>" --armor --output "$i"{.asc,}
done
```
5. Upload `<path-to-achow101>` (now with signatures) to `guix.sigs`
-----
After this change, output directories will now include a `SHA256SUMS.part` fragment, created immediately after a successful build:
```
output
└── x86_64-w64-mingw32
├── bitcoin-4e069f7589da-win64-debug.zip
├── bitcoin-4e069f7589da-win64-setup-unsigned.exe
├── bitcoin-4e069f7589da-win64.zip
├── bitcoin-4e069f7589da-win-unsigned.tar.gz
└── SHA256SUMS.part
```
These `SHA256SUMS.part` fragments look something like:
```
3ebd7262b1a0a5bb757fef1f70e7e14033c70f98c059bc4dbfee5d1992b25825 dist-archive/bitcoin-4e069f7589da.tar.gz
def2e7d3de5ab3e3f955344e75151df4f33713f9101f5295bd13c9375bdf633b x86_64-w64-mingw32/bitcoin-4e069f7589da-win64-debug.zip
643049fe3ee4a4e83a1739607e67b11b7c9b1a66208a6f35a9ff634ba795500e x86_64-w64-mingw32/bitcoin-4e069f7589da-win64-setup-unsigned.exe
a247a1ccec0ccc2e138c648284bd01f6a761f2d8d6d07d91b5b4a6670ec3f288 x86_64-w64-mingw32/bitcoin-4e069f7589da-win-unsigned.tar.gz
fab76a836dcc592e39c04fd2396696633fb6eb56e39ecbf6c909bd173ed4280c x86_64-w64-mingw32/bitcoin-4e069f7589da-win64.zip
```
Meaning that they are valid `SHA256SUMS` files when `sha256sum --check`'d at the `guix-build-*/output` directory level
When `guix-attest` is invoked, these `SHA256SUMS.part` files are combined and sorted (by `-k2`, `LC_ALL=C`) to create:
1. `noncodesigned.SHA256SUMS` for a manifest of all non-codesigned outputs, and
3. `all.SHA256SUMS` for a manifest of all outputs including non-codesigned outputs
Then both files are signed, resulting in the following `guix.sigs` hierarchy:
```
4e069f7589da/
└── dongcarl
├── all.SHA256SUMS
├── all.SHA256SUMS.asc
├── noncodesigned.SHA256SUMS
└── noncodesigned.SHA256SUMS.asc
```
ACKs for top commit:
achow101:
ACK e2c40a4ed5272d72fea997bd936fba28bb753226
hebasto:
ACK e2c40a4ed5272d72fea997bd936fba28bb753226, tested on Linux Mint 20.1 (x86_64) with and w/o `NO_SIGN=1`. Changes in `contrib/guix/libexec/codesign.sh` and `contrib/guix/guix-verify` are reviewed only.
Tree-SHA512: 618aacefb0eb6595735a9ab6a98ea6598fce65f9ccf33fa1e7ef93bf140c0f6cfc16e34870c6aa3e4777dd3f004b92a82a994141879870141742df948ec59c1f
|
||
|
fanquake
|
ff34f3e3b4 |
Merge bitcoin/bitcoin#22075: guix: Misc leftover usability improvements
108a6be92adc1e80839d90b552e72b8142140f6c guix: Check for disk space availability before building (Carl Dong) d7dec89091ee4a456ff64ad7ce675ae6813668f1 guix: Remove dest if OUTDIR mv fails (Carl Dong) Pull request description: There seems to be some corner cases that can be hit when guix scripts unexpectedly fail in the middle of operation, see: https://gnusha.org/bitcoin-builds/2021-05-24.log - Perform an early disk space check for `guix-build` - Overwrite existing output directory after a successful build (the existing one might be malformed), and cleanup output directory if the `mv` somehow fails ACKs for top commit: laanwj: Tested ACK 108a6be92adc1e80839d90b552e72b8142140f6c achow101: ACK 108a6be92adc1e80839d90b552e72b8142140f6c Tree-SHA512: cf6438317da40bf55714cd2d8cce859b3d435cc66cabefe8d4a53552d7880966acfe84ffe8fadf1c80e368ae6b037992258a6d409df85ffc6ce8bf780e98e2e5 |
||
|
fanquake
|
0cc419752b |
Merge bitcoin/bitcoin#21654: build, qt: Make Qt rcc output always deterministic
a58868d201cb6d263aa552815f7f86562c1ca9a5 build: Makes rcc output always deterministic (Hennadii Stepanov) Pull request description: The Qt Resource Compiler ([rcc](https://doc.qt.io/qt-5/rcc.html)) has a command-line option `--format-version` which has the [default value](https://code.qt.io/cgit/qt/qtbase.git/tree/src/tools/rcc/main.cpp?h=5.12.10#n172) 2. The only difference from `--format-version 1` is adding a [last modified timestamp](https://code.qt.io/cgit/qt/qtbase.git/tree/src/tools/rcc/rcc.cpp?h=5.12.10#n207) to the output file ([credits](https://github.com/bitcoin/bitcoin/pull/21654#issuecomment-819198228) to **fanquake**). That, in turn, forces us to use `QT_RCC_SOURCE_DATE_OVERRIDE=1` to get deterministic builds (#13732). This change makes rcc output always deterministic by using `--format-version 1` option that makes usage of the `QT_RCC_SOURCE_DATE_OVERRIDE` needless. --- Also it improves interaction with ccache: On master (f6c44e999b7d1d9a0de5d678ac8f1679aa271f65): ``` $ make && make clean && ccache --zero-stats && make && ccache --show-stats ... cache directory /home/hebasto/.ccache primary config /home/hebasto/.ccache/ccache.conf secondary config (readonly) /etc/ccache.conf stats updated Sun Apr 11 15:45:43 2021 stats zeroed Sun Apr 11 15:45:05 2021 cache hit (direct) 638 cache hit (preprocessed) 0 cache miss 1 cache hit rate 99.84 % called for link 10 cleanups performed 0 files in cache 20023 cache size 13.2 GB max cache size 15.0 GB ``` The missed file is always `qt/libbitcoinqt_a-qrc_bitcoin_locale.o`. With this PR: ``` $ make && make clean && ccache --zero-stats && make && ccache --show-stats ... cache directory /home/hebasto/.ccache primary config /home/hebasto/.ccache/ccache.conf secondary config (readonly) /etc/ccache.conf stats updated Sun Apr 11 15:28:46 2021 stats zeroed Sun Apr 11 15:28:21 2021 cache hit (direct) 639 cache hit (preprocessed) 0 cache miss 0 cache hit rate 100.00 % called for link 10 cleanups performed 0 files in cache 20012 cache size 13.2 GB max cache size 15.0 GB ``` ACKs for top commit: fanquake: ACK a58868d201cb6d263aa552815f7f86562c1ca9a5 Tree-SHA512: 52f4a3267f41883d13025c0de79b6da22e92d60c729e01b986935c6812bbfe7fadc40b742bd715bfdf09df94af6838d4fbbe8208c6123f366108e38c8e1121c5 |
||
|
W. J. van der Laan
|
f95b802504 |
Merge bitcoin/bitcoin#21462: guix: Add guix-{attest,verify} scripts
d420e5c1c015f58d07aca4d6a805086488f74d03 guix-attest: Avoid incomplete sigdirs with ERR traps (Carl Dong)
feda2c8e3180cb983c35976d4440cea23a155b7f guix: Skip attesting to dist-archive (Carl Dong)
d522d8006b891eccd7901faf391f9c041ddf8e38 guix: Attest to inputs in inputs.SHA256SUMS (Carl Dong)
f9e2960c018103be756a7f8a506816b49d662514 guix: Construct $OUTDIR in ${DISTSRC}/output (Carl Dong)
022abc85fc7e711a900fed8e5071919a151c0a63 guix: Minor quoting fix in libexec/build.sh (Carl Dong)
c83c4fa5b78aef33bba36b3a0d273422297bd630 guix-attest: Allow skipping GPG signing with NO_SIGN (Carl Dong)
0e1c2e448c25568f276e4f022128870c76ca216b guix-attest: Use ascii-armor signatures (Carl Dong)
b5fd89c4c89136007429688601ce4fa497f5f09e guix-attest: Only use cross-platform flags for find+xargs (Carl Dong)
5926432ba68ba154df6c8eaa74adb18cc0123167 guix: Add guix-verify script (Carl Dong)
30daf76a97c57a5f74c8dad1da282dcc0ff8b3fb guix: Add guix-attest script (Carl Dong)
Pull request description:
Adds replacements for `gsign` and `gverify`.
Personally I'm not a big fan of using the word "sign" as it's been used to refer to both codesigning and GPG signing.
ACKs for top commit:
laanwj:
Code review and tested ACK d420e5c1c015f58d07aca4d6a805086488f74d03
Tree-SHA512: 93d82d201f4596eaea0e3825aa55b013dfb91790e6ccee79893833d37921513d7b4e735f0641103e1e2ea8308abe4cb6218b73160924708802f2e0e3f7f6caf1
|
||
|
W. J. van der Laan
|
67efaa8808 |
Merge bitcoin/bitcoin#21799: guix: Use gcc-8 across the board
c90f6e51094a1ba4fb2aab35b78f23b6fda645d0 guix: Consistently use gcc-8 for $HOST (Carl Dong) Pull request description: Only non-base commit is the last commit: |
||
|
fanquake
|
c62a180d2d |
Merge #21655: build, qt: No longer need to set QT_RCC_TEST=1 for determinism
c799a19b4bb8d0cc5ffd9b49746b5b267745c9b5 build, qt: No longer need to set QT_RCC_TEST=1 for determinism (Hennadii Stepanov)
Pull request description:
The Qt Resource Compiler (rcc) output order relies on [`QHash`](https://doc.qt.io/qt-5/qhash.html):
> This randomization of `QHash` is enabled by default. Even though programs should never depend on a particular `QHash` ordering, there may be situations where you temporarily need deterministic behavior, for example for debugging or regression testing. To disable the randomization, define the environment variable `QT_HASH_SEED` to have the value 0.
Since #3620 we use `QT_RCC_TEST=1` to achieve a deterministic output.
Since Qt 5.3.1 hash seeding is disabled for all of the bootstrapped tools, including rcc. Therefore, `QT_RCC_TEST=1` is no longer needed.
See commit [5283a6c87beac5a43f612786fefd6e43f2c70bf6](
|
||
|
W. J. van der Laan
|
dfd1045284 |
Merge #21375: guix: Misc feedback-based fixes + hier restructuring
7476b46f1893a4858616d2a8456a7c43238851ed guix: Build dmg as a static binary (Carl Dong)
06d6cf6784421290e6235fe8684d5e08ed6f1b62 depends: libdmg-hfsplus: Skip CMake RPATH patching (Carl Dong)
65176ab5730dff34466caaecdd292625ef8294fc guix: Remove codesign_allocate+pagestuff from unsigned tarball (Carl Dong)
ca85679eb43b8375a95d82101977829d08fb1e1b guix: Use clang-toolchain instead of clang (Carl Dong)
1aec0eda8fd31a57b0621eea616398017c2ead98 guix: Fallback to local build for substitute-enabled Guix users (Carl Dong)
1742f8e12d163852df09575e03edcd3db73198ee guix: Add early health check for guix-daemon (Carl Dong)
c1ae726a13ecfa5e7e9fdc3030a8110b8bb263f8 guix: More thoroughly control native toolchain (Carl Dong)
39741128d3775d198dbee34dc827353bfd18acd8 guix: Supply --link-profile (Carl Dong)
d55a1056ee565afed64e42d6f6efb6b0adc5599b guix: Add troubleshooting documentation entries (Carl Dong)
7f401c953f8bb3574cec48561e13ef3b47dedc6e guix: Adapt guix-build to prelude, restructure hier (Carl Dong)
4eccf063b252bfe256cf72d363a24cf0183e926e guix: Remove guix-build.sh filename extension (Carl Dong)
7753357a7bae98ec775c707b9dec4cea1e945802 guix: Add source-able bash prelude and utils (Carl Dong)
e5b49a01f5d0f631e7f08f86ca8a2c2b8213319f guix: Create windeploy inside distsrc-* (Carl Dong)
3e9982ab3877eb8fe0a8c0cb3d847ac0913c7336 contrib: Silence git-describe when looking for tag (Carl Dong)
d5a71e97853ea9e1b879e8c76bfb01d4bef33172 guix: Use --cores instead of --max-jobs (Carl Dong)
Pull request description:
This PR addresses a few hiccups encountered by the brave souls who've been experimenting with the Guix scripts:
- Resolves confusion between `--cores=` and `--max-jobs=`
- `guix`'s `--cores=` actually corresponds to make's `--jobs=`, so let's just control `--cores=` with our overridable env var
- `git-describe` will scream `fatal: no tag exactly matches '<hash>'` when looking for a tag, but we don't care, so silence that
- `windeploy/unsigned` should be inside `distsrc-*` and created idempotently (sorry I know this one annoyed people)
- Add troubleshooting documentation to `README.md`
- Add early health check for `guix-daemon` in case user forgot to start a `guix-daemon`
- Depending on configuration, a `--fallback` flag may be needed to tell Guix to not fail if substitutes fail but fallback to building locally
- `codesign_allocate` and `pagestuff` are now unnecessary for codesigning as we're now using `signapple`
A few robustness changes are also included:
- We supply the `--link-profile` flag, as some Guix packages may expect the profile to be available under `$HOME/.guix-profile`
- We now clear and manually set all toolchain-related env vars (e.g. `C*_INCLUDE_PATH`) ourselves, after patching a Qt::moc bug
- We use the native `clang-toolchain` package for darwin builds instead of `clang`, lining up with all our other toolchain packages.
Finally, we restructure the guix building hierarchy such that it looks something like:
```
guix-build-<short-hash-or-version-tag>
├── distsrc-<short-hash-or-version-tag>-${HOST}
│ ├── contrib
│ ├── depends
│ ├── src
│ └── ...
├── distsrc-<short-hash-or-version-tag>-...
└── output
├── dist-archive
│ └── bitcoin-<short-hash-or-version-tag>.tar.gz
├── *-linux-*
│ ├── bitcoin-<short-hash-or-version-tag>-*-linux-*-debug.tar.gz
│ └── bitcoin-<short-hash-or-version-tag>-*-linux-*.tar.gz
├── x86_64-apple-darwin18
│ ├── bitcoin-<short-hash-or-version-tag>-osx64.tar.gz
│ ├── bitcoin-<short-hash-or-version-tag>-osx-unsigned.dmg
│ └── bitcoin-<short-hash-or-version-tag>-osx-unsigned.tar.gz
└── x86_64-w64-mingw32
├── bitcoin-<short-hash-or-version-tag>-win64-debug.zip
├── bitcoin-<short-hash-or-version-tag>-win64-setup-unsigned.exe
├── bitcoin-<short-hash-or-version-tag>-win64.zip
└── bitcoin-<short-hash-or-version-tag>-win-unsigned.tar.gz
```
Separating guix builds by their version identifier (basically namespacing them) allows us to change the layout in the future without worry about potential naming conflicts.
ACKs for top commit:
sipa:
ACK 7476b46f1893a4858616d2a8456a7c43238851ed
laanwj:
ACK 7476b46f1893a4858616d2a8456a7c43238851ed
Tree-SHA512: 0e899aa941aafdf552b2a7e8a08131ee9283180bbef7334439e2461a02aa7235ab7b9ca9c149b80fc5d0a9f4bbd35bc80fcee26197c0836ba8eaf2d86ffa0386
|
||
|
Wladimir J. van der Laan
|
5fda1ea15a |
Merge #21337: guix: Update darwin native packages dependencies
c967fb7fb97a86fc434912d0fff621f9f50dfe80 guix: Remove libcap from manifest (Hennadii Stepanov) 7bbb409314c63750a9731dedaaf6dd6ecd17d988 guix: Update darwin native packages dependencies (Hennadii Stepanov) Pull request description: It is a #20470 follow up. ACKs for top commit: fanquake: ACK c967fb7fb97a86fc434912d0fff621f9f50dfe80 Tree-SHA512: 66ce05770f578ba61a44c58747c5a2669f425a989ed987838058bd86e3b49e342ac5a4f8852fc49f2b3a86b58fb6a340fdf3e34c1fc19bdab910729febba4bc7 |
||
|
fanquake
|
2c4bf310df |
partial Merge #21325: lint: Fix spelling errors in comments
fbbb2d4fc13971c98c83a51635166ac532e71a32 lint: Fix spelling errors in comments (fyquah) Pull request description: Found some spelling errors while running spelling linter https://github.com/bitcoin/bitcoin/pull/21245 This PR fixes them. ACKs for top commit: fanquake: ACK fbbb2d4fc13971c98c83a51635166ac532e71a32 - I thought we just fixed all of these. Tree-SHA512: 95525040001f94e899b778c616cb66ebafb679dff88835b66fccf6349d8eb942d6b7374c536a44e393f13156bce9a32ed57e6a82bb02074d2b3cddb2696addb2 |
||
|
Wladimir J. van der Laan
|
a305fc4c51 |
Merge #21272: guix: Passthrough SDK_PATH into container
13a9fd11a507fd3398bc2c0a0575bdc81579243f guix: Passthrough SDK_PATH into container (Carl Dong) Pull request description: This is a usability improvement for Guix builders so that they don't have to extract the Xcode tarball into `depends/SDKs` every time. Inspiration: https://github.com/bitcoin/bitcoin/pull/21089#issuecomment-778639698 ACKs for top commit: laanwj: Tested ACK 13a9fd11a507fd3398bc2c0a0575bdc81579243f Tree-SHA512: 63392d537e48a0da9f0ee04a929613b139bef1ac5643187871c9ea5376afd2a3d95df0f5e0950ae0eccd2813b166667be98401e5a248ae9c187fe4e84e54d427 |
||
|
fanquake
|
0b78348aa1 |
Merge #21271: guix: Explicitly set umask in build container
d98f4593cf00ab2973f8113e30506861b24383bc guix: Explicitly set umask in build container (Carl Dong) Pull request description: Opened as a separate PR to fix non-reproducibility found through testing here: https://github.com/bitcoin/bitcoin/pull/21089#issuecomment-783549633 Many thanks to everyone who helped find this! ACKs for top commit: laanwj: ACK d98f4593cf00ab2973f8113e30506861b24383bc fanquake: ACK d98f4593cf00ab2973f8113e30506861b24383bc - I'm seeing matching hashes. Tree-SHA512: ea339c3902f2f4dea32e8ef5cc675a1df0679530881260ae999aaaf7339d5b12c46e01e58677cbb079f33e573ad105e2b443a835f3e944ef8e943a25f83027f1 |
||
|
fanquake
|
9c8f5f71dc |
partial Merge #21089: guix: Add support for powerpc64{,le}
95990b9f3278360b63e79d6975af4ab5009c66ba guix: Update conservative space requirements (Carl Dong)
5e6df1132656995ce5b9ce279d5a9808ea52ab32 guix: Add support for powerpc64{,le} (Carl Dong)
Pull request description:
```
The new time-machine commit contains a few small changes that make the
powerpc cross-toolchain work.
```
See this compare to review my custom patches to Guix:
|
||
|
Wladimir J. van der Laan
|
cd0d7d2991 |
Merge #21088: guix: Jump forwards in time-machine and adapt
d02076b8852d8faae95cee6e3de434460c07412a guix: Jump forwards in time-machine and adapt (Carl Dong)
f8ca8c5c28d3050b780e67d47a50ac65fc2dc3ad guix: Supply --keep-failed for debugging (Carl Dong)
Pull request description:
```
The new time-machine commit is Guix v1.2.0 with a yet-unupstreamed patch
for NSIS.
A few important changes:
1. Guix switched back from using CPATH to C{,PLUS}_INCLUDE_PATH as the
way to indicate #include search paths.
2. GCC's library is now split into a separate output, whereas before it
was included in the default output. This means that our gcc toolchain
packages need to propagate that output.
3. A few package versions were bumped
```
See this compare to review my custom patches to Guix: https://github.com/dongcarl/guix/compare/version-1.2.0...7d6bd44da57926e0d4af25eba723a61c82beef98
ACKs for top commit:
laanwj:
ACK d02076b8852d8faae95cee6e3de434460c07412a
Tree-SHA512: 896d5bf1b6e5fda2f0106013c568c119bbbb86cb31a8c0a22432bada9b7da51678b96374bf8fd7c15353698ba47ac9dd39874d40c39001281471db7c78bf1705
|
||
|
fanquake
|
2f3b44289c |
Merge #21087: guix: Passthrough BASE_CACHE into container
901f54321b386258a1682423160bfdfa35ea4c39 guix: Passthrough BASE_CACHE into container (Carl Dong)
Pull request description:
This allows depends-built packages to be cached.
ACKs for top commit:
MarcoFalke:
Approach ACK 901f54321b386258a1682423160bfdfa35ea4c39
fanquake:
ACK 901f54321b386258a1682423160bfdfa35ea4c39
Tree-SHA512: 464815f41fc081d7956bec84380668834b6ee6751c7a3d56daad6e1fc91e582de4bbdd1a89f399b1136f2adc4d9941517cfe4db694f0ee5bf59bf2f44fc6fda0
|
||
|
fanquake
|
89ff5e3ef8 |
Merge #21116: build: Disable --disable-fuzz-binary for gitian/guix builds
cb151b797aeabb376d37ef058aee34c06d36d487 build: Disable --disable-fuzz-binary for guix builds (Hennadii Stepanov)
fd7caae35fd3f83175247a79f7573e374779a851 build: Disable --disable-fuzz-binary for gitian builds (Hennadii Stepanov)
Pull request description:
Fuzz binary is not shipped to users.
This PR saves hundreds MB of the disk space for containers.
ACKs for top commit:
MarcoFalke:
review ACK cb151b797aeabb376d37ef058aee34c06d36d487
fanquake:
ACK cb151b797aeabb376d37ef058aee34c06d36d487
Tree-SHA512: 858e3816576c307b47915bb05de79a28029beaef8835c01f1bd6a764a0cf7f7f63ef8c2dc2c5944cb36cc9f4788d9b0590b8a5dda96940167252ba371cdbd078
|
||
|
fanquake
|
a6e7dda55b |
Merge #17920: guix: Build support for macOS
f1694757ddbcb3635213b085e864851e285c8c12 guix: Fix typo (Carl Dong)
771c4b98a8693eee642f2b118b3193fe6e022291 guix: README: Add darwin HOSTS entry (Carl Dong)
8dbf18cb1d3260d34ba822ceb12e67b1f124ea13 guix: Check for macOS SDK before building anything (Carl Dong)
34b23f597ec52efb795d72e9e5620712d0010edd guix: Set ZERO_AR_DATE for darwin build determinism (Carl Dong)
f3835dc6a3732dcd4afbb5987f84dc27f2bf55af build: Make xorrisofs reproducible with -volume_date (Carl Dong)
c9eb4cf3a0f81bfd72f06fd43b5610f0a4f5e804 guix: Add support for darwin builds (Carl Dong)
37fe73a092b08fe9d7ce636a1021429de6cda757 build: Add var printing target to src/Makefile.am (Carl Dong)
Pull request description:
This PR brings our Guix builds on par with Gitian in terms of supported architectures.
Reviewers: if you run a build, please submit:
```
find output/ -type f -print0 | env LC_ALL=C sort -z | xargs -r0 sha256sum
```
So that we can compare hashes and ensure reproducibility!
ACKs for top commit:
fanquake:
ACK f1694757ddbcb3635213b085e864851e285c8c12 - I think we can make some small usability improvements, but this is ok to merge now.
Tree-SHA512: 4af2b71654a9736467dcc681d10601c6eee37800d7847011a50585455b67b55d61742ca5604585f310a2fd75335b674e5e27dfb5169cb2f26e112aa4c411d8be
|
||
|
fanquake
|
d4d19b539b |
Merge #20619: guix: Quality of life improvements
570e43fe72e13e0a82e25f7145704f62b2c2cc52 guix: Print build params inside/outside of container (Carl Dong)
2f9d1fdde66f4713351905ec73487e5288d20f8f guix: Move DISTSRC determination to guix-build.sh (Carl Dong)
0b7cd07bb56baa112ffa596fb23a905871031a36 guix: Move OUTDIR determination+creation to guix-build.sh (Carl Dong)
d27ff8b86aa66acec63b5713912bd4ad9470e66f guix: Add more sanity checks to guix-build.sh (Carl Dong)
57f95331464f097261c63fd1b6040536c58a03fa guix: Add section headings to guix-build.sh (Carl Dong)
38b7b2ed72b1f0f57bd9800c7fbb7b7c98a20ed0 genbuild: Specify rev-parse length (Carl Dong)
036dc740da3239cdcc13e0f299ab95b456f7118b docs: Point to contrib/guix/README.md in doc/guix.md (Carl Dong)
34f0fda2d31d2ada632ca1165b82aebdfd342efe guix: Small updates to README wording (Carl Dong)
402e3a5b1ed9de7057ce9955ea792ad1c2b9f2b5 guix: Update HOSTS README entry for new architectures (Carl Dong)
cfa7ceb21b14d1fa24c2541bf242a0ed539b9e1b guix: Remove README development environment section (Carl Dong)
93b6a8544a03d13733ca2ef769f76df587ad86c8 guix: Add ADDITIONAL_GUIX_{COMMON,TIMEMACHINE}_FLAGS options (Carl Dong)
0f31e24703e25698d2d41fb54e30ec75a4a80943 guix: Add SUBSTITUTE_URLS option (Carl Dong)
444fcfca907d46cfeb52001599966cce25bdf54e guix: Make guix honor MAX_JOBS setting (Carl Dong)
Pull request description:
After live-demo-ing a Guix build (which completed successfully!) on achow101's stream, I realized there were a few quality of life improvements which can be made to improve the user experience of our Guix build process. Here are a few of them.
Notable changes:
1. When `MAX_JOBS` is specified, both `guix time-machine` and `guix environment` will now build up to `MAX_JOBS` packages at a time when creating the build environment
2. The instructions for using substitutes were incorrect, and has now been replaced with a `SUBSTITUTE_URLS` environment variable, which works well with shell's IFS splitting rules
3. New `ADDITIONAL_GUIX_{COMMON,TIMEMACHINE}_FLAGS` options, for more granular customization of the build process.
4. README cleanup
ACKs for top commit:
fanquake:
ACK 570e43fe72e13e0a82e25f7145704f62b2c2cc52 - lets move this forward.
Tree-SHA512: 4e8ab560522ade5efb5e8736aec0fb1a3f19ae9deb586c1ab87020816876f3f466a950b3f8c04d9fa1d072ae5ee780038c5c9063577049bdd9db17978e11c328
|
||
|
Wladimir J. van der Laan
|
de430e69aa |
Merge #20318: build: Ensure source tarball has leading directory name
faa2f06f5eaf8578873495f44603ee74d7a1abf4 scripted-diff: [build] Ensure source tarball has leading directory name (MarcoFalke) Pull request description: This has been fixed in 0.20, so it needs to be fixed on master as well to avoid a regression #18945 ACKs for top commit: laanwj: ACK faa2f06f5eaf8578873495f44603ee74d7a1abf4 hebasto: ACK faa2f06f5eaf8578873495f44603ee74d7a1abf4, tested gitian builds only. promag: ACK faa2f06f5eaf8578873495f44603ee74d7a1abf4. Tree-SHA512: e3b025c29c45b025002abc35262bb5d771f6cbd807f1c256c477c243685e93cd43ad9f642b38e3cf218590912abe6ea0ddfec3bfbef36f99080aad74ed6cc0af |
||
|
fanquake
|
a3ceaea97e |
Merge #18958: guix: Make V=1 more powerful for debugging
f852761aec81ed23c7b9e4546c08d1ef303f2507 guix: Add clarifying documentation for V env var (Carl Dong)
85f4a4b0822e3aa10310c4623eff719f301e9263 guix: Make V=1 more powerful for debugging (Carl Dong)
Pull request description:
```
- Print commands in both unexpanded and expanded forms
- Set VERBOSE=1 for CMake
```
Ping MarcoFalke hopefully you use `V=1` already for the Guix builds on DrahtBot?
ACKs for top commit:
fanquake:
ACK f852761aec81ed23c7b9e4546c08d1ef303f2507. Ran a Windows Guix build and compared the output from master and this PR when using `V=1`. i.e `HOSTS=x86_64-w64-mingw32 PATH="/root/.config/guix/current/bin${PATH:+:}$PATH" V=1 ./contrib/guix/guix-build.sh`.
Tree-SHA512: 8bc466fa7b869618bbd5a0a91c6b23d4785009289f8dfb93b0349317463a9ab9ece128c72436e02a0819722a63e703100aed15807867a716fda891292fcb9d9d
|
||
|
fanquake
|
700d46b26d |
Merge #18741: guix: Make source tarball using git-archive
bfe1ba2f5b36056e0c41edf8206b93d3d83098df rel-builds: Specify core.abbrev for git-rev-parse (Carl Dong) 27e63e01cce368d67092de8f0c736927d6f6aa69 build: Accomodate makensis v2.x (Carl Dong) 1f2c39a30e0f82046c7aecddfda3eb99cb536816 guix: Remove logical cores requirement (Carl Dong) a4f6ffa71e335d4b2a6bf525b7f416968f9cd9f7 lint: Also enable source statements for non-gitian (Carl Dong) d256f91cb1b0d6ff5170106b99b0266cbe51f5a2 rel-builds: Directly deploy win installer to OUTDIR (Carl Dong) fa791da02f9684e3fd554b687fb692ae6a23d65a nsis: Specify OutFile path only once (Carl Dong) 14701604d0904bc5bbf1c67de08f8ee6d3215523 guix: Expose GIT_COMMON_DIR in container as readonly (Carl Dong) f5a6ac4f48b18f93050d77bcb23f9cf45ec34647 guix: Make source tarball using git-archive (Carl Dong) 395c1137f630dc495ffb2752a23bc1dfd470ee53 gitian: Limit sourced script to just assignments (Carl Dong) Pull request description: Based on: #18556 Related: https://github.com/bitcoin/bitcoin/pull/17595#discussion_r399728721 ACKs for top commit: fanquake: ACK bfe1ba2f5b36056e0c41edf8206b93d3d83098df - I agree with Carl, and am going to merge this. I'd like for Linux Guix builds to be working again, and we can rebase #18818. Tree-SHA512: c87ada7e3de17ca0b692a91029b86573442ded5780fc081c214773f6b374a0cdbeaf6f6898c36669c2e247ee32aa7f82defb1180f8decac52c65f0c140f18674 |
||
|
fanquake
|
87de9c8f5c |
Merge #17929: build: add linker optimisation flags to gitian & guix (Linux)
f2b5b0a3b48f9241971c14238040048f8b630811 build: add linker optimization flags to guix (fanquake) b8b050a8d642e38c81d1e451750c2c8db92fee5e build: add linker optimization flags to gitian descriptors (fanquake) Pull request description: This PR adds `-Wl,O2` to our gitian and guix LDFLAGS. This makes the linker perform certain optimisations (and is different from LTO). Any -O argument will enable optimizations in GNU ld. We can use -O2 here, as this matches our compile flags. Note that this would also enable additional optimizations if using the lld or gold linkers, when compared to -O0. A nice writeup + diagrams of some of these optimizations is available here: http://lwn.net/Articles/192624/. #### master ```bash # bitcoind Histogram for `.gnu.hash' bucket list length (total of 3 buckets) Length Number % of total Coverage 0 1 ( 33.3%) 0.0% 1 0 ( 0.0%) 0.0% 2 1 ( 33.3%) 40.0% 3 1 ( 33.3%) 100.0% ``` ```bash # bitcoin-qt Histogram for `.gnu.hash' bucket list length (total of 3 buckets) Length Number % of total Coverage 0 0 ( 0.0%) 0.0% 1 1 ( 33.3%) 10.0% 2 0 ( 0.0%) 10.0% 3 0 ( 0.0%) 10.0% 4 1 ( 33.3%) 50.0% 5 1 ( 33.3%) 100.0% ``` #### this PR: ```bash # bitcoind Histogram for `.gnu.hash' bucket list length (total of 8 buckets) Length Number % of total Coverage 0 3 ( 37.5%) 0.0% 1 5 ( 62.5%) 100.0% ``` ```bash # bitcoin-qt Histogram for `.gnu.hash' bucket list length (total of 19 buckets) Length Number % of total Coverage 0 9 ( 47.4%) 0.0% 1 10 ( 52.6%) 100.0% ``` #### GNU ld -O > If level is a numeric values greater than zero ld optimizes the output. This might take significantly longer and therefore probably should only be enabled for the final binary. At the moment this option only affects ELF shared library generation. Future releases of the linker may make more use of this option. Also currently there is no difference in the linker’s behaviour for different non-zero values of this option. Again this may change with future releases. #### lld -O > Optimize output file size ACKs for top commit: dongcarl: ACK f2b5b0a3b48f9241971c14238040048f8b630811 laanwj: ACK f2b5b0a3b48f9241971c14238040048f8b630811 Tree-SHA512: e53f3a4338317dbec65d3a93b57b5a6204aabdf9ac82d99447847a3c8627facc53c58c2cf947376f13edd979fc8129a80f18d9ebeccd191a576c83f1dad5c513 |
||
|
fanquake
|
dff599acff |
Merge #17595: guix: Enable building for x86_64-w64-mingw32 target
a35e3235891d35daa167116cc70340140e883f06 guix: Appease travis. (Carl Dong) 0b66d22da5f53640e22f05adf880782c613e6d0f guix: Use gcc-9 for mingw-w64 instead of 8 (Carl Dong) ba0b99bdd613ba7f17c6247ece3001e1b44759a3 guix: Don't set MINGW_HAS_SECURE_API CFLAG in depends (Carl Dong) 93439a71eda49fb69f1e82966a23a946733aa6fa guix: Bump to upstream commit with mingw-w64 changes (Carl Dong) 35a96792dda9e78165b1598aeac7b2ab759e7be5 guix: Check mingw symbols, improve SSP fix docs (Carl Dong) 449d8fe25bbe25daacfc67aa89ca32b0a3254c5a guix: Expand on INT trap message (Carl Dong) 3f1f03c67a8e9edf487f08d272adb18b0a3942c8 guix: Spelling fixes (Carl Dong) ff821dd2a1c600488d11e7d9a20e9179ecc9144b guix: Reinstate make-ssp-fixed-gcc (Carl Dong) 360a9e0ad50a36ec79a1a160dbed3966689fd41c guix: Bump time-machine for mingw-w64 patches (Carl Dong) 93e41b7e3b54c17fd1b4c61ee95fc0dc2827e954 guix: Use gcc-8 for mingw-w64 instead of 7 (Carl Dong) ef4f7e4c45c60a69406134122f091c77c6ef740f guix: Set the well-known timezone env var (Carl Dong) acf4b3b3b5accf60a19441a0298ef27001b78e72 guix: Make x86_64-w64-mingw32 builds reproducible (Carl Dong) c4cce00eac691625b78b92f7dba0b7f57def19e5 guix: Remove dead links from README. (Carl Dong) df953a4c9a6143f45864757b706c88b6fa70545a guix: Appease shellcheck. (Carl Dong) 91897c95e191d293eb27d8af15cbeafc5b8f3895 guix: Improve guix-build.sh documentation (Carl Dong) 570d769c6c59b9f6d1a2b95b2ed60432cb33b3ba guix: Build support for Windows (Carl Dong) Pull request description: ~~Based on: https://github.com/bitcoin/bitcoin/pull/16519~~ Based on: #17933 (Time Machines are... shall we say... superior 😁) This PR allows us to perform Guix builds for the `x86_64-w64-mingw32` target. We do this _without_ splitting up the build script like we do in Gitian by using this newfangled alien technology called `case` statements. (This is WIP and might be changed to `if` statements soon) ACKs for top commit: fanquake: ACK a35e3235891d35daa167116cc70340140e883f06 2/3 Tree-SHA512: c471951c23eb2cda919a71285d8b8f2580cb20f09d5db17b53e13dbd8813e01b3e7a83ea848e4913fd0f2bc12c6c133c5f76b54e65c0d89fed4dfd2e0be19875 |
||
|
fanquake
|
9eb0995a88 |
Merge #16810: guix: Remove ssp spec file hack
0065ead5ebdad9f743d89ee2558c93253a597967 contrib: guix: Remove ssp spec file hack (Carl Dong)
0093a5869a30797ccb139d0cb5a0427d3f3c2d94 contrib: guix: More robust search paths, add checks (Carl Dong)
Pull request description:
See commit messages for more details
ACKs for top commit:
fanquake:
ACK 0065ead5ebdad9f743d89ee2558c93253a597967
Tree-SHA512: fde04005fb31cd4b75b80da4936a7c394f63f0b3bdcc33c20c99e05604a63efd9c850a8d097030ff0bf4b4e83f1a9997fc4621ce291ebcecd8397893447600a7
|
||
|
Wladimir J. van der Laan
|
aec7441ac2 |
Merge #15277: contrib: Enable building in Guix containers
751549b52a9a4cd27389d807ae67f02bbb39cd7f contrib: guix: Additional clarifications re: substitutes (Carl Dong) cd3e947f50db7cfe05c05b368c25742193729a62 contrib: guix: Various improvements. (Carl Dong) 8dff3e48a9e03299468ed3b342642f01f70da9db contrib: guix: Clarify SOURCE_DATE_EPOCH. (Carl Dong) 3e80ec3ea9691c7c89173de922a113e643fe976b contrib: Add deterministic Guix builds. (Carl Dong) Pull request description: ~~**This post is kept updated as this project progresses. Use this [latest update link](https://github.com/bitcoin/bitcoin/pull/15277#issuecomment-497303718) to see what's new.**~~ Please read the `README.md`. ----- ### Guix Introduction This PR enables building bitcoin in Guix containers. [Guix](https://www.gnu.org/software/guix/manual/en/html_node/Features.html) is a transactional package manager much like Nix, but unlike Nix, it has more of a focus on [bootstrappability](https://www.gnu.org/software/guix/manual/en/html_node/Bootstrapping.html) and [reproducibility](https://www.gnu.org/software/guix/blog/tags/reproducible-builds/) which are attractive for security-sensitive projects like bitcoin. ### Guix Build Walkthrough Please read the `README.md`. [Old instructions no. 4](https://github.com/bitcoin/bitcoin/pull/15277#issuecomment-497303718) [Old instructions no. 3](https://github.com/bitcoin/bitcoin/pull/15277#issuecomment-493827011) [Old instructions no. 2](https://github.com/bitcoin/bitcoin/pull/15277#issuecomment-471658439) <details> <summary>Old instructions no. 1</summary> In this PR, we define a Guix [manifest](https://www.gnu.org/software/guix/manual/en/html_node/Invoking-guix-package.html#profile_002dmanifest) in `contrib/guix/manifest.scm`, which declares what packages we want in our environment. We can then invoke ``` guix environment --manifest=contrib/guix/manifest.scm --container --pure --no-grafts --no-substitutes ``` To have Guix: 1. Build an environment containing the packages we defined in our `contrib/guix/manifest.scm` manifest from the Guix bootstrap binaries (see [bootstrappability](https://www.gnu.org/software/guix/manual/en/html_node/Bootstrapping.html) for more details). 2. Start a container with that environment that has no network access, and no access to the host's filesystem except to the `pwd` that it was started in. 3. Drop you into a shell in that container. > Note: if you don't want to wait hours for Guix to build the entire world from scratch, you can eliminate the `--no-substitutes` option to have Guix download from available binary sources. Note that this convenience doesn't necessarily compromise your security, as you can check that a package was built correctly after the fact using `guix build --check <packagename>` Therefore, we can perform a build of bitcoin much like in Gitian by invoking the following: ``` make -C depends -j"$(nproc)" download && \ cat contrib/guix/build.sh | guix environment --manifest=contrib/guix/manifest.scm --container --pure --no-grafts --no-substitutes ``` We don't include `make -C depends -j"$(nproc)" download` inside `contrib/guix/build.sh` because `contrib/guix/build.sh` is run inside the container, which has no network access (which is a good thing). </details> ### Rationale I believe that this represents a substantial improvement for the "supply chain security" of bitcoin because: 1. We no longer have to rely on Ubuntu for our build environment for our releases ([oh the horror]( |