dash/contrib/guix/manifest.scm

668 lines
26 KiB
Scheme
Raw Normal View History

Merge #15277: contrib: Enable building in Guix containers 751549b52a9a4cd27389d807ae67f02bbb39cd7f contrib: guix: Additional clarifications re: substitutes (Carl Dong) cd3e947f50db7cfe05c05b368c25742193729a62 contrib: guix: Various improvements. (Carl Dong) 8dff3e48a9e03299468ed3b342642f01f70da9db contrib: guix: Clarify SOURCE_DATE_EPOCH. (Carl Dong) 3e80ec3ea9691c7c89173de922a113e643fe976b contrib: Add deterministic Guix builds. (Carl Dong) Pull request description: ~~**This post is kept updated as this project progresses. Use this [latest update link](https://github.com/bitcoin/bitcoin/pull/15277#issuecomment-497303718) to see what's new.**~~ Please read the `README.md`. ----- ### Guix Introduction This PR enables building bitcoin in Guix containers. [Guix](https://www.gnu.org/software/guix/manual/en/html_node/Features.html) is a transactional package manager much like Nix, but unlike Nix, it has more of a focus on [bootstrappability](https://www.gnu.org/software/guix/manual/en/html_node/Bootstrapping.html) and [reproducibility](https://www.gnu.org/software/guix/blog/tags/reproducible-builds/) which are attractive for security-sensitive projects like bitcoin. ### Guix Build Walkthrough Please read the `README.md`. [Old instructions no. 4](https://github.com/bitcoin/bitcoin/pull/15277#issuecomment-497303718) [Old instructions no. 3](https://github.com/bitcoin/bitcoin/pull/15277#issuecomment-493827011) [Old instructions no. 2](https://github.com/bitcoin/bitcoin/pull/15277#issuecomment-471658439) <details> <summary>Old instructions no. 1</summary> In this PR, we define a Guix [manifest](https://www.gnu.org/software/guix/manual/en/html_node/Invoking-guix-package.html#profile_002dmanifest) in `contrib/guix/manifest.scm`, which declares what packages we want in our environment. We can then invoke ``` guix environment --manifest=contrib/guix/manifest.scm --container --pure --no-grafts --no-substitutes ``` To have Guix: 1. Build an environment containing the packages we defined in our `contrib/guix/manifest.scm` manifest from the Guix bootstrap binaries (see [bootstrappability](https://www.gnu.org/software/guix/manual/en/html_node/Bootstrapping.html) for more details). 2. Start a container with that environment that has no network access, and no access to the host's filesystem except to the `pwd` that it was started in. 3. Drop you into a shell in that container. > Note: if you don't want to wait hours for Guix to build the entire world from scratch, you can eliminate the `--no-substitutes` option to have Guix download from available binary sources. Note that this convenience doesn't necessarily compromise your security, as you can check that a package was built correctly after the fact using `guix build --check <packagename>` Therefore, we can perform a build of bitcoin much like in Gitian by invoking the following: ``` make -C depends -j"$(nproc)" download && \ cat contrib/guix/build.sh | guix environment --manifest=contrib/guix/manifest.scm --container --pure --no-grafts --no-substitutes ``` We don't include `make -C depends -j"$(nproc)" download` inside `contrib/guix/build.sh` because `contrib/guix/build.sh` is run inside the container, which has no network access (which is a good thing). </details> ### Rationale I believe that this represents a substantial improvement for the "supply chain security" of bitcoin because: 1. We no longer have to rely on Ubuntu for our build environment for our releases ([oh the horror](https://github.com/bitcoin/bitcoin/blob/72bd4ab867e3be0d8410403d9641c08288d343e3/contrib/gitian-descriptors/gitian-linux.yml#L10)), because Guix builds everything about the container, we can perform this on almost any Linux distro/system. 2. It is now much easier to determine what trusted binaries are in our supply chain, and even make a nice visualization! (see [bootstrappability](https://www.gnu.org/software/guix/manual/en/html_node/Bootstrapping.html)). 3. There is active effort among Guix folks to minimize the number of trusted binaries even further. OriansJ's [stage0](https://github.com/oriansj/stage0), and janneke's [Mes](https://www.gnu.org/software/mes/) all aim to achieve [reduced binary boostrap](http://joyofsource.com/reduced-binary-seed-bootstrap.html) for Guix. In fact, I believe if OriansJ gets his way, we will end up some day with only a single trusted binary: hex0 (a ~500 byte self-hosting hex assembler). ### Steps to Completion - [x] Successfully build bitcoin inside the Guix environment - [x] Make `check-symbols` pass - [x] Do the above but without nasty hacks - [x] Solve some of the more innocuous hacks - [ ] Make it cross-compile (HELP WANTED HERE) - [x] Linux - [x] x86_64-linux-gnu - [x] i686-linux-gnu - [x] aarch64-linux-gnu - [x] arm-linux-gnueabihf - [x] riscv64-linux-gnu - [ ] OS X - [ ] x86_64-apple-darwin14 - [ ] Windows - [ ] x86_64-w64-mingw32 - [ ] Maybe make importer for depends syntax - [ ] Document build process for future releases - [ ] Extra: Pin the revision of Guix that we build with with Guix [inferiors](https://www.gnu.org/software/guix/manual/en/html_node/Inferiors.html) ### Help Wanted [Old content no. 3](https://github.com/bitcoin/bitcoin/pull/15277#issuecomment-483318210) [Old content no. 2](https://github.com/bitcoin/bitcoin/pull/15277#issuecomment-471658439) <details> <summary>Old content no. 1</summary> As of now, the command described above to perform a build of bitcoin a lot like Gitian works, but fails at the `check-symbols` stage. This is because a few dynamic libraries are linked in that shouldn't be. Here's what `ldd src/bitcoind` looks like when built in a Guix container: ``` linux-vdso.so.1 (0x00007ffcc2d90000) libdl.so.2 => /gnu/store/h90vnqw0nwd0hhm1l5dgxsdrigddfmq4-glibc-2.28/lib/libdl.so.2 (0x00007fb7eda09000) librt.so.1 => /gnu/store/h90vnqw0nwd0hhm1l5dgxsdrigddfmq4-glibc-2.28/lib/librt.so.1 (0x00007fb7ed9ff000) libstdc++.so.6 => /gnu/store/4sqps8dczv3g7rwbdibfz6rf5jlk7w90-gcc-5.5.0-lib/lib/libstdc++.so.6 (0x00007fb7ed87c000) libpthread.so.0 => /gnu/store/h90vnqw0nwd0hhm1l5dgxsdrigddfmq4-glibc-2.28/lib/libpthread.so.0 (0x00007fb7ed85b000) libm.so.6 => /gnu/store/h90vnqw0nwd0hhm1l5dgxsdrigddfmq4-glibc-2.28/lib/libm.so.6 (0x00007fb7ed6da000) libgcc_s.so.1 => /gnu/store/4sqps8dczv3g7rwbdibfz6rf5jlk7w90-gcc-5.5.0-lib/lib/libgcc_s.so.1 (0x00007fb7ed6bf000) libc.so.6 => /gnu/store/h90vnqw0nwd0hhm1l5dgxsdrigddfmq4-glibc-2.28/lib/libc.so.6 (0x00007fb7ed506000) /gnu/store/h90vnqw0nwd0hhm1l5dgxsdrigddfmq4-glibc-2.28/lib/ld-linux-x86-64.so.2 => /usr/lib64/ld-linux-x86-64.so.2 (0x00007fb7ee3a0000) ``` And here's what it looks in one of our releases: ``` linux-vdso.so.1 (0x00007ffff52cd000) libpthread.so.0 => /usr/lib/libpthread.so.0 (0x00007f87726b4000) librt.so.1 => /usr/lib/librt.so.1 (0x00007f87726aa000) libm.so.6 => /usr/lib/libm.so.6 (0x00007f8772525000) libgcc_s.so.1 => /usr/lib/libgcc_s.so.1 (0x00007f877250b000) libc.so.6 => /usr/lib/libc.so.6 (0x00007f8772347000) /lib64/ld-linux-x86-64.so.2 => /usr/lib64/ld-linux-x86-64.so.2 (0x00007f8773392000) ``` ~~I suspect it is because my script does not apply the gitian-input patches [described in the release process](https://github.com/bitcoin/bitcoin/blob/master/doc/release-process.md#fetch-and-create-inputs-first-time-or-when-dependency-versions-change) but there is no description as to how these patches are applied.~~ It might also be something else entirely. Edit: It is something else. It appears that the gitian inputs are only used by [`gitian-win-signer.yml`](https://github.com/bitcoin/bitcoin/blob/d6e700e40f861ddd6743f4d13f0d6f6bc19093c2/contrib/gitian-descriptors/gitian-win-signer.yml#L14) </details> ### How to Help 1. Install Guix on your distro either [from source](https://www.gnu.org/software/guix/manual/en/html_node/Requirements.html) or perform a [binary installation](https://www.gnu.org/software/guix/manual/en/html_node/Binary-Installation.html#Binary-Installation) 2. Try out my branch and the command described above! ACKs for top commit: MarcoFalke: Thanks for the replies. ACK 751549b52a9a4cd27389d807ae67f02bbb39cd7f laanwj: ACK 751549b52a9a4cd27389d807ae67f02bbb39cd7f Tree-SHA512: 50e6ab58c6bda9a67125b6271daf7eff0ca57d0efa8941ed3cd951e5bf78b31552fc5e537b1e1bcf2d3cc918c63adf19d685aa117a0f851024dc67e697890a8d
2019-07-12 19:23:30 +02:00
(use-modules (gnu)
(gnu packages)
(gnu packages autotools)
(gnu packages base)
(gnu packages bash)
Merge bitcoin/bitcoin#21239: guix: Add codesignature attachment support for osx+win ee883201cf134952284632e9e9ae72bf1c8c792f guix: repro: Sort find output in libtool for gcc-8 (Carl Dong) ee0a67c32a8861eab650bf8894af06807578eba0 codesigning: Use SHA256 as digest for osslsigncode (Windows) (Carl Dong) 38eb91eb0616ed6dbe34c23e11588d130fef07f8 guix: Add codesigning functionality (Carl Dong) bac2690e6f683fcedb883fe1d32f3c33c628a141 guix: Package codesigning tools (Carl Dong) 0a2176d47767972e4cd5ed302c1dbeedece1708b guix: Reindent existing manifest.scm (Carl Dong) c090a3e9238ba2df07875b4708e908d8dca4ed9b Makefile.am: use APP_DIST_DIR instead of hard-coding dist (Carl Dong) Pull request description: This is the last PR before we reach feature-parity with the Gitian process! Note: I tried using the `Makefile` inside the distsrc to make the dmg instead of manually listing out the commands, but `make` seems to want to re-make a lot of other files which broke the dmg. The workflow looks something like this: 1. `env [ FOO=bar... ] ./contrib/guix/guix-build` (add additional env vars as necessary) 2. Codesigners only: 1. Copy `guix-build-<short-id>/output/x86_64-apple-darwin18/bitcoin-<short-id>-osx-unsigned.tar.gz` and `guix-build-<short-id>/output/x86_64-w64-mingw32/bitcoin-<short-id>-win-unsigned.tar.gz` to signing computer 2. Codesign with `./detached-sig-create.sh` inside the tarball 3. Upload contents of `signature-{osx,win}.tar.gz` to https://github.com/bitcoin-core/bitcoin-detached-sigs (as a new tag) 3. Checkout new tag for `bitcoin-core/bitcoin-detached-sigs` with the detached signatures 4. `env [ FOO=bar... ] DETACHED_SIGS_REPO=<path/to/bitcoin-detached-sigs> ./contrib/guix/guix-codesign` (modify env vars as necessary) 5. Make sure `guix.sigs` is cloned and updated 6. `env GUIX_SIGS_REPO=<path/to/guix.sigs> SIGNER=0x96AB007F1A7ED999=dongcarl ./contrib/guix/guix-attest` (modify env vars as necessary) 7. Commit your new signatures and SHA256SUMS in `guix.sigs` 8. Optionally, after there are multiple signatures in `guix.sigs`: `env GUIX_SIGS_REPO=<path/to/guix.sigs> ./contrib/guix/guix-verify` ACKs for top commit: laanwj: Tested ACK ee883201cf134952284632e9e9ae72bf1c8c792f achow101: ACK ee883201cf134952284632e9e9ae72bf1c8c792f Tree-SHA512: e812a07a5f19f900600c70cb9c717769ef544a6c0c12760b5558b76b6b37df863257f3dbf38b0757e6e06e334470267e94c9f2bdbc27409d6837b1a0bfc6acbc
2021-05-24 15:31:05 +02:00
(gnu packages certs)
(gnu packages cdrom)
Merge #15277: contrib: Enable building in Guix containers 751549b52a9a4cd27389d807ae67f02bbb39cd7f contrib: guix: Additional clarifications re: substitutes (Carl Dong) cd3e947f50db7cfe05c05b368c25742193729a62 contrib: guix: Various improvements. (Carl Dong) 8dff3e48a9e03299468ed3b342642f01f70da9db contrib: guix: Clarify SOURCE_DATE_EPOCH. (Carl Dong) 3e80ec3ea9691c7c89173de922a113e643fe976b contrib: Add deterministic Guix builds. (Carl Dong) Pull request description: ~~**This post is kept updated as this project progresses. Use this [latest update link](https://github.com/bitcoin/bitcoin/pull/15277#issuecomment-497303718) to see what's new.**~~ Please read the `README.md`. ----- ### Guix Introduction This PR enables building bitcoin in Guix containers. [Guix](https://www.gnu.org/software/guix/manual/en/html_node/Features.html) is a transactional package manager much like Nix, but unlike Nix, it has more of a focus on [bootstrappability](https://www.gnu.org/software/guix/manual/en/html_node/Bootstrapping.html) and [reproducibility](https://www.gnu.org/software/guix/blog/tags/reproducible-builds/) which are attractive for security-sensitive projects like bitcoin. ### Guix Build Walkthrough Please read the `README.md`. [Old instructions no. 4](https://github.com/bitcoin/bitcoin/pull/15277#issuecomment-497303718) [Old instructions no. 3](https://github.com/bitcoin/bitcoin/pull/15277#issuecomment-493827011) [Old instructions no. 2](https://github.com/bitcoin/bitcoin/pull/15277#issuecomment-471658439) <details> <summary>Old instructions no. 1</summary> In this PR, we define a Guix [manifest](https://www.gnu.org/software/guix/manual/en/html_node/Invoking-guix-package.html#profile_002dmanifest) in `contrib/guix/manifest.scm`, which declares what packages we want in our environment. We can then invoke ``` guix environment --manifest=contrib/guix/manifest.scm --container --pure --no-grafts --no-substitutes ``` To have Guix: 1. Build an environment containing the packages we defined in our `contrib/guix/manifest.scm` manifest from the Guix bootstrap binaries (see [bootstrappability](https://www.gnu.org/software/guix/manual/en/html_node/Bootstrapping.html) for more details). 2. Start a container with that environment that has no network access, and no access to the host's filesystem except to the `pwd` that it was started in. 3. Drop you into a shell in that container. > Note: if you don't want to wait hours for Guix to build the entire world from scratch, you can eliminate the `--no-substitutes` option to have Guix download from available binary sources. Note that this convenience doesn't necessarily compromise your security, as you can check that a package was built correctly after the fact using `guix build --check <packagename>` Therefore, we can perform a build of bitcoin much like in Gitian by invoking the following: ``` make -C depends -j"$(nproc)" download && \ cat contrib/guix/build.sh | guix environment --manifest=contrib/guix/manifest.scm --container --pure --no-grafts --no-substitutes ``` We don't include `make -C depends -j"$(nproc)" download` inside `contrib/guix/build.sh` because `contrib/guix/build.sh` is run inside the container, which has no network access (which is a good thing). </details> ### Rationale I believe that this represents a substantial improvement for the "supply chain security" of bitcoin because: 1. We no longer have to rely on Ubuntu for our build environment for our releases ([oh the horror](https://github.com/bitcoin/bitcoin/blob/72bd4ab867e3be0d8410403d9641c08288d343e3/contrib/gitian-descriptors/gitian-linux.yml#L10)), because Guix builds everything about the container, we can perform this on almost any Linux distro/system. 2. It is now much easier to determine what trusted binaries are in our supply chain, and even make a nice visualization! (see [bootstrappability](https://www.gnu.org/software/guix/manual/en/html_node/Bootstrapping.html)). 3. There is active effort among Guix folks to minimize the number of trusted binaries even further. OriansJ's [stage0](https://github.com/oriansj/stage0), and janneke's [Mes](https://www.gnu.org/software/mes/) all aim to achieve [reduced binary boostrap](http://joyofsource.com/reduced-binary-seed-bootstrap.html) for Guix. In fact, I believe if OriansJ gets his way, we will end up some day with only a single trusted binary: hex0 (a ~500 byte self-hosting hex assembler). ### Steps to Completion - [x] Successfully build bitcoin inside the Guix environment - [x] Make `check-symbols` pass - [x] Do the above but without nasty hacks - [x] Solve some of the more innocuous hacks - [ ] Make it cross-compile (HELP WANTED HERE) - [x] Linux - [x] x86_64-linux-gnu - [x] i686-linux-gnu - [x] aarch64-linux-gnu - [x] arm-linux-gnueabihf - [x] riscv64-linux-gnu - [ ] OS X - [ ] x86_64-apple-darwin14 - [ ] Windows - [ ] x86_64-w64-mingw32 - [ ] Maybe make importer for depends syntax - [ ] Document build process for future releases - [ ] Extra: Pin the revision of Guix that we build with with Guix [inferiors](https://www.gnu.org/software/guix/manual/en/html_node/Inferiors.html) ### Help Wanted [Old content no. 3](https://github.com/bitcoin/bitcoin/pull/15277#issuecomment-483318210) [Old content no. 2](https://github.com/bitcoin/bitcoin/pull/15277#issuecomment-471658439) <details> <summary>Old content no. 1</summary> As of now, the command described above to perform a build of bitcoin a lot like Gitian works, but fails at the `check-symbols` stage. This is because a few dynamic libraries are linked in that shouldn't be. Here's what `ldd src/bitcoind` looks like when built in a Guix container: ``` linux-vdso.so.1 (0x00007ffcc2d90000) libdl.so.2 => /gnu/store/h90vnqw0nwd0hhm1l5dgxsdrigddfmq4-glibc-2.28/lib/libdl.so.2 (0x00007fb7eda09000) librt.so.1 => /gnu/store/h90vnqw0nwd0hhm1l5dgxsdrigddfmq4-glibc-2.28/lib/librt.so.1 (0x00007fb7ed9ff000) libstdc++.so.6 => /gnu/store/4sqps8dczv3g7rwbdibfz6rf5jlk7w90-gcc-5.5.0-lib/lib/libstdc++.so.6 (0x00007fb7ed87c000) libpthread.so.0 => /gnu/store/h90vnqw0nwd0hhm1l5dgxsdrigddfmq4-glibc-2.28/lib/libpthread.so.0 (0x00007fb7ed85b000) libm.so.6 => /gnu/store/h90vnqw0nwd0hhm1l5dgxsdrigddfmq4-glibc-2.28/lib/libm.so.6 (0x00007fb7ed6da000) libgcc_s.so.1 => /gnu/store/4sqps8dczv3g7rwbdibfz6rf5jlk7w90-gcc-5.5.0-lib/lib/libgcc_s.so.1 (0x00007fb7ed6bf000) libc.so.6 => /gnu/store/h90vnqw0nwd0hhm1l5dgxsdrigddfmq4-glibc-2.28/lib/libc.so.6 (0x00007fb7ed506000) /gnu/store/h90vnqw0nwd0hhm1l5dgxsdrigddfmq4-glibc-2.28/lib/ld-linux-x86-64.so.2 => /usr/lib64/ld-linux-x86-64.so.2 (0x00007fb7ee3a0000) ``` And here's what it looks in one of our releases: ``` linux-vdso.so.1 (0x00007ffff52cd000) libpthread.so.0 => /usr/lib/libpthread.so.0 (0x00007f87726b4000) librt.so.1 => /usr/lib/librt.so.1 (0x00007f87726aa000) libm.so.6 => /usr/lib/libm.so.6 (0x00007f8772525000) libgcc_s.so.1 => /usr/lib/libgcc_s.so.1 (0x00007f877250b000) libc.so.6 => /usr/lib/libc.so.6 (0x00007f8772347000) /lib64/ld-linux-x86-64.so.2 => /usr/lib64/ld-linux-x86-64.so.2 (0x00007f8773392000) ``` ~~I suspect it is because my script does not apply the gitian-input patches [described in the release process](https://github.com/bitcoin/bitcoin/blob/master/doc/release-process.md#fetch-and-create-inputs-first-time-or-when-dependency-versions-change) but there is no description as to how these patches are applied.~~ It might also be something else entirely. Edit: It is something else. It appears that the gitian inputs are only used by [`gitian-win-signer.yml`](https://github.com/bitcoin/bitcoin/blob/d6e700e40f861ddd6743f4d13f0d6f6bc19093c2/contrib/gitian-descriptors/gitian-win-signer.yml#L14) </details> ### How to Help 1. Install Guix on your distro either [from source](https://www.gnu.org/software/guix/manual/en/html_node/Requirements.html) or perform a [binary installation](https://www.gnu.org/software/guix/manual/en/html_node/Binary-Installation.html#Binary-Installation) 2. Try out my branch and the command described above! ACKs for top commit: MarcoFalke: Thanks for the replies. ACK 751549b52a9a4cd27389d807ae67f02bbb39cd7f laanwj: ACK 751549b52a9a4cd27389d807ae67f02bbb39cd7f Tree-SHA512: 50e6ab58c6bda9a67125b6271daf7eff0ca57d0efa8941ed3cd951e5bf78b31552fc5e537b1e1bcf2d3cc918c63adf19d685aa117a0f851024dc67e697890a8d
2019-07-12 19:23:30 +02:00
(gnu packages check)
(gnu packages cmake)
Merge #15277: contrib: Enable building in Guix containers 751549b52a9a4cd27389d807ae67f02bbb39cd7f contrib: guix: Additional clarifications re: substitutes (Carl Dong) cd3e947f50db7cfe05c05b368c25742193729a62 contrib: guix: Various improvements. (Carl Dong) 8dff3e48a9e03299468ed3b342642f01f70da9db contrib: guix: Clarify SOURCE_DATE_EPOCH. (Carl Dong) 3e80ec3ea9691c7c89173de922a113e643fe976b contrib: Add deterministic Guix builds. (Carl Dong) Pull request description: ~~**This post is kept updated as this project progresses. Use this [latest update link](https://github.com/bitcoin/bitcoin/pull/15277#issuecomment-497303718) to see what's new.**~~ Please read the `README.md`. ----- ### Guix Introduction This PR enables building bitcoin in Guix containers. [Guix](https://www.gnu.org/software/guix/manual/en/html_node/Features.html) is a transactional package manager much like Nix, but unlike Nix, it has more of a focus on [bootstrappability](https://www.gnu.org/software/guix/manual/en/html_node/Bootstrapping.html) and [reproducibility](https://www.gnu.org/software/guix/blog/tags/reproducible-builds/) which are attractive for security-sensitive projects like bitcoin. ### Guix Build Walkthrough Please read the `README.md`. [Old instructions no. 4](https://github.com/bitcoin/bitcoin/pull/15277#issuecomment-497303718) [Old instructions no. 3](https://github.com/bitcoin/bitcoin/pull/15277#issuecomment-493827011) [Old instructions no. 2](https://github.com/bitcoin/bitcoin/pull/15277#issuecomment-471658439) <details> <summary>Old instructions no. 1</summary> In this PR, we define a Guix [manifest](https://www.gnu.org/software/guix/manual/en/html_node/Invoking-guix-package.html#profile_002dmanifest) in `contrib/guix/manifest.scm`, which declares what packages we want in our environment. We can then invoke ``` guix environment --manifest=contrib/guix/manifest.scm --container --pure --no-grafts --no-substitutes ``` To have Guix: 1. Build an environment containing the packages we defined in our `contrib/guix/manifest.scm` manifest from the Guix bootstrap binaries (see [bootstrappability](https://www.gnu.org/software/guix/manual/en/html_node/Bootstrapping.html) for more details). 2. Start a container with that environment that has no network access, and no access to the host's filesystem except to the `pwd` that it was started in. 3. Drop you into a shell in that container. > Note: if you don't want to wait hours for Guix to build the entire world from scratch, you can eliminate the `--no-substitutes` option to have Guix download from available binary sources. Note that this convenience doesn't necessarily compromise your security, as you can check that a package was built correctly after the fact using `guix build --check <packagename>` Therefore, we can perform a build of bitcoin much like in Gitian by invoking the following: ``` make -C depends -j"$(nproc)" download && \ cat contrib/guix/build.sh | guix environment --manifest=contrib/guix/manifest.scm --container --pure --no-grafts --no-substitutes ``` We don't include `make -C depends -j"$(nproc)" download` inside `contrib/guix/build.sh` because `contrib/guix/build.sh` is run inside the container, which has no network access (which is a good thing). </details> ### Rationale I believe that this represents a substantial improvement for the "supply chain security" of bitcoin because: 1. We no longer have to rely on Ubuntu for our build environment for our releases ([oh the horror](https://github.com/bitcoin/bitcoin/blob/72bd4ab867e3be0d8410403d9641c08288d343e3/contrib/gitian-descriptors/gitian-linux.yml#L10)), because Guix builds everything about the container, we can perform this on almost any Linux distro/system. 2. It is now much easier to determine what trusted binaries are in our supply chain, and even make a nice visualization! (see [bootstrappability](https://www.gnu.org/software/guix/manual/en/html_node/Bootstrapping.html)). 3. There is active effort among Guix folks to minimize the number of trusted binaries even further. OriansJ's [stage0](https://github.com/oriansj/stage0), and janneke's [Mes](https://www.gnu.org/software/mes/) all aim to achieve [reduced binary boostrap](http://joyofsource.com/reduced-binary-seed-bootstrap.html) for Guix. In fact, I believe if OriansJ gets his way, we will end up some day with only a single trusted binary: hex0 (a ~500 byte self-hosting hex assembler). ### Steps to Completion - [x] Successfully build bitcoin inside the Guix environment - [x] Make `check-symbols` pass - [x] Do the above but without nasty hacks - [x] Solve some of the more innocuous hacks - [ ] Make it cross-compile (HELP WANTED HERE) - [x] Linux - [x] x86_64-linux-gnu - [x] i686-linux-gnu - [x] aarch64-linux-gnu - [x] arm-linux-gnueabihf - [x] riscv64-linux-gnu - [ ] OS X - [ ] x86_64-apple-darwin14 - [ ] Windows - [ ] x86_64-w64-mingw32 - [ ] Maybe make importer for depends syntax - [ ] Document build process for future releases - [ ] Extra: Pin the revision of Guix that we build with with Guix [inferiors](https://www.gnu.org/software/guix/manual/en/html_node/Inferiors.html) ### Help Wanted [Old content no. 3](https://github.com/bitcoin/bitcoin/pull/15277#issuecomment-483318210) [Old content no. 2](https://github.com/bitcoin/bitcoin/pull/15277#issuecomment-471658439) <details> <summary>Old content no. 1</summary> As of now, the command described above to perform a build of bitcoin a lot like Gitian works, but fails at the `check-symbols` stage. This is because a few dynamic libraries are linked in that shouldn't be. Here's what `ldd src/bitcoind` looks like when built in a Guix container: ``` linux-vdso.so.1 (0x00007ffcc2d90000) libdl.so.2 => /gnu/store/h90vnqw0nwd0hhm1l5dgxsdrigddfmq4-glibc-2.28/lib/libdl.so.2 (0x00007fb7eda09000) librt.so.1 => /gnu/store/h90vnqw0nwd0hhm1l5dgxsdrigddfmq4-glibc-2.28/lib/librt.so.1 (0x00007fb7ed9ff000) libstdc++.so.6 => /gnu/store/4sqps8dczv3g7rwbdibfz6rf5jlk7w90-gcc-5.5.0-lib/lib/libstdc++.so.6 (0x00007fb7ed87c000) libpthread.so.0 => /gnu/store/h90vnqw0nwd0hhm1l5dgxsdrigddfmq4-glibc-2.28/lib/libpthread.so.0 (0x00007fb7ed85b000) libm.so.6 => /gnu/store/h90vnqw0nwd0hhm1l5dgxsdrigddfmq4-glibc-2.28/lib/libm.so.6 (0x00007fb7ed6da000) libgcc_s.so.1 => /gnu/store/4sqps8dczv3g7rwbdibfz6rf5jlk7w90-gcc-5.5.0-lib/lib/libgcc_s.so.1 (0x00007fb7ed6bf000) libc.so.6 => /gnu/store/h90vnqw0nwd0hhm1l5dgxsdrigddfmq4-glibc-2.28/lib/libc.so.6 (0x00007fb7ed506000) /gnu/store/h90vnqw0nwd0hhm1l5dgxsdrigddfmq4-glibc-2.28/lib/ld-linux-x86-64.so.2 => /usr/lib64/ld-linux-x86-64.so.2 (0x00007fb7ee3a0000) ``` And here's what it looks in one of our releases: ``` linux-vdso.so.1 (0x00007ffff52cd000) libpthread.so.0 => /usr/lib/libpthread.so.0 (0x00007f87726b4000) librt.so.1 => /usr/lib/librt.so.1 (0x00007f87726aa000) libm.so.6 => /usr/lib/libm.so.6 (0x00007f8772525000) libgcc_s.so.1 => /usr/lib/libgcc_s.so.1 (0x00007f877250b000) libc.so.6 => /usr/lib/libc.so.6 (0x00007f8772347000) /lib64/ld-linux-x86-64.so.2 => /usr/lib64/ld-linux-x86-64.so.2 (0x00007f8773392000) ``` ~~I suspect it is because my script does not apply the gitian-input patches [described in the release process](https://github.com/bitcoin/bitcoin/blob/master/doc/release-process.md#fetch-and-create-inputs-first-time-or-when-dependency-versions-change) but there is no description as to how these patches are applied.~~ It might also be something else entirely. Edit: It is something else. It appears that the gitian inputs are only used by [`gitian-win-signer.yml`](https://github.com/bitcoin/bitcoin/blob/d6e700e40f861ddd6743f4d13f0d6f6bc19093c2/contrib/gitian-descriptors/gitian-win-signer.yml#L14) </details> ### How to Help 1. Install Guix on your distro either [from source](https://www.gnu.org/software/guix/manual/en/html_node/Requirements.html) or perform a [binary installation](https://www.gnu.org/software/guix/manual/en/html_node/Binary-Installation.html#Binary-Installation) 2. Try out my branch and the command described above! ACKs for top commit: MarcoFalke: Thanks for the replies. ACK 751549b52a9a4cd27389d807ae67f02bbb39cd7f laanwj: ACK 751549b52a9a4cd27389d807ae67f02bbb39cd7f Tree-SHA512: 50e6ab58c6bda9a67125b6271daf7eff0ca57d0efa8941ed3cd951e5bf78b31552fc5e537b1e1bcf2d3cc918c63adf19d685aa117a0f851024dc67e697890a8d
2019-07-12 19:23:30 +02:00
(gnu packages commencement)
(gnu packages compression)
(gnu packages cross-base)
Merge bitcoin/bitcoin#21239: guix: Add codesignature attachment support for osx+win ee883201cf134952284632e9e9ae72bf1c8c792f guix: repro: Sort find output in libtool for gcc-8 (Carl Dong) ee0a67c32a8861eab650bf8894af06807578eba0 codesigning: Use SHA256 as digest for osslsigncode (Windows) (Carl Dong) 38eb91eb0616ed6dbe34c23e11588d130fef07f8 guix: Add codesigning functionality (Carl Dong) bac2690e6f683fcedb883fe1d32f3c33c628a141 guix: Package codesigning tools (Carl Dong) 0a2176d47767972e4cd5ed302c1dbeedece1708b guix: Reindent existing manifest.scm (Carl Dong) c090a3e9238ba2df07875b4708e908d8dca4ed9b Makefile.am: use APP_DIST_DIR instead of hard-coding dist (Carl Dong) Pull request description: This is the last PR before we reach feature-parity with the Gitian process! Note: I tried using the `Makefile` inside the distsrc to make the dmg instead of manually listing out the commands, but `make` seems to want to re-make a lot of other files which broke the dmg. The workflow looks something like this: 1. `env [ FOO=bar... ] ./contrib/guix/guix-build` (add additional env vars as necessary) 2. Codesigners only: 1. Copy `guix-build-<short-id>/output/x86_64-apple-darwin18/bitcoin-<short-id>-osx-unsigned.tar.gz` and `guix-build-<short-id>/output/x86_64-w64-mingw32/bitcoin-<short-id>-win-unsigned.tar.gz` to signing computer 2. Codesign with `./detached-sig-create.sh` inside the tarball 3. Upload contents of `signature-{osx,win}.tar.gz` to https://github.com/bitcoin-core/bitcoin-detached-sigs (as a new tag) 3. Checkout new tag for `bitcoin-core/bitcoin-detached-sigs` with the detached signatures 4. `env [ FOO=bar... ] DETACHED_SIGS_REPO=<path/to/bitcoin-detached-sigs> ./contrib/guix/guix-codesign` (modify env vars as necessary) 5. Make sure `guix.sigs` is cloned and updated 6. `env GUIX_SIGS_REPO=<path/to/guix.sigs> SIGNER=0x96AB007F1A7ED999=dongcarl ./contrib/guix/guix-attest` (modify env vars as necessary) 7. Commit your new signatures and SHA256SUMS in `guix.sigs` 8. Optionally, after there are multiple signatures in `guix.sigs`: `env GUIX_SIGS_REPO=<path/to/guix.sigs> ./contrib/guix/guix-verify` ACKs for top commit: laanwj: Tested ACK ee883201cf134952284632e9e9ae72bf1c8c792f achow101: ACK ee883201cf134952284632e9e9ae72bf1c8c792f Tree-SHA512: e812a07a5f19f900600c70cb9c717769ef544a6c0c12760b5558b76b6b37df863257f3dbf38b0757e6e06e334470267e94c9f2bdbc27409d6837b1a0bfc6acbc
2021-05-24 15:31:05 +02:00
(gnu packages curl)
Merge #15277: contrib: Enable building in Guix containers 751549b52a9a4cd27389d807ae67f02bbb39cd7f contrib: guix: Additional clarifications re: substitutes (Carl Dong) cd3e947f50db7cfe05c05b368c25742193729a62 contrib: guix: Various improvements. (Carl Dong) 8dff3e48a9e03299468ed3b342642f01f70da9db contrib: guix: Clarify SOURCE_DATE_EPOCH. (Carl Dong) 3e80ec3ea9691c7c89173de922a113e643fe976b contrib: Add deterministic Guix builds. (Carl Dong) Pull request description: ~~**This post is kept updated as this project progresses. Use this [latest update link](https://github.com/bitcoin/bitcoin/pull/15277#issuecomment-497303718) to see what's new.**~~ Please read the `README.md`. ----- ### Guix Introduction This PR enables building bitcoin in Guix containers. [Guix](https://www.gnu.org/software/guix/manual/en/html_node/Features.html) is a transactional package manager much like Nix, but unlike Nix, it has more of a focus on [bootstrappability](https://www.gnu.org/software/guix/manual/en/html_node/Bootstrapping.html) and [reproducibility](https://www.gnu.org/software/guix/blog/tags/reproducible-builds/) which are attractive for security-sensitive projects like bitcoin. ### Guix Build Walkthrough Please read the `README.md`. [Old instructions no. 4](https://github.com/bitcoin/bitcoin/pull/15277#issuecomment-497303718) [Old instructions no. 3](https://github.com/bitcoin/bitcoin/pull/15277#issuecomment-493827011) [Old instructions no. 2](https://github.com/bitcoin/bitcoin/pull/15277#issuecomment-471658439) <details> <summary>Old instructions no. 1</summary> In this PR, we define a Guix [manifest](https://www.gnu.org/software/guix/manual/en/html_node/Invoking-guix-package.html#profile_002dmanifest) in `contrib/guix/manifest.scm`, which declares what packages we want in our environment. We can then invoke ``` guix environment --manifest=contrib/guix/manifest.scm --container --pure --no-grafts --no-substitutes ``` To have Guix: 1. Build an environment containing the packages we defined in our `contrib/guix/manifest.scm` manifest from the Guix bootstrap binaries (see [bootstrappability](https://www.gnu.org/software/guix/manual/en/html_node/Bootstrapping.html) for more details). 2. Start a container with that environment that has no network access, and no access to the host's filesystem except to the `pwd` that it was started in. 3. Drop you into a shell in that container. > Note: if you don't want to wait hours for Guix to build the entire world from scratch, you can eliminate the `--no-substitutes` option to have Guix download from available binary sources. Note that this convenience doesn't necessarily compromise your security, as you can check that a package was built correctly after the fact using `guix build --check <packagename>` Therefore, we can perform a build of bitcoin much like in Gitian by invoking the following: ``` make -C depends -j"$(nproc)" download && \ cat contrib/guix/build.sh | guix environment --manifest=contrib/guix/manifest.scm --container --pure --no-grafts --no-substitutes ``` We don't include `make -C depends -j"$(nproc)" download` inside `contrib/guix/build.sh` because `contrib/guix/build.sh` is run inside the container, which has no network access (which is a good thing). </details> ### Rationale I believe that this represents a substantial improvement for the "supply chain security" of bitcoin because: 1. We no longer have to rely on Ubuntu for our build environment for our releases ([oh the horror](https://github.com/bitcoin/bitcoin/blob/72bd4ab867e3be0d8410403d9641c08288d343e3/contrib/gitian-descriptors/gitian-linux.yml#L10)), because Guix builds everything about the container, we can perform this on almost any Linux distro/system. 2. It is now much easier to determine what trusted binaries are in our supply chain, and even make a nice visualization! (see [bootstrappability](https://www.gnu.org/software/guix/manual/en/html_node/Bootstrapping.html)). 3. There is active effort among Guix folks to minimize the number of trusted binaries even further. OriansJ's [stage0](https://github.com/oriansj/stage0), and janneke's [Mes](https://www.gnu.org/software/mes/) all aim to achieve [reduced binary boostrap](http://joyofsource.com/reduced-binary-seed-bootstrap.html) for Guix. In fact, I believe if OriansJ gets his way, we will end up some day with only a single trusted binary: hex0 (a ~500 byte self-hosting hex assembler). ### Steps to Completion - [x] Successfully build bitcoin inside the Guix environment - [x] Make `check-symbols` pass - [x] Do the above but without nasty hacks - [x] Solve some of the more innocuous hacks - [ ] Make it cross-compile (HELP WANTED HERE) - [x] Linux - [x] x86_64-linux-gnu - [x] i686-linux-gnu - [x] aarch64-linux-gnu - [x] arm-linux-gnueabihf - [x] riscv64-linux-gnu - [ ] OS X - [ ] x86_64-apple-darwin14 - [ ] Windows - [ ] x86_64-w64-mingw32 - [ ] Maybe make importer for depends syntax - [ ] Document build process for future releases - [ ] Extra: Pin the revision of Guix that we build with with Guix [inferiors](https://www.gnu.org/software/guix/manual/en/html_node/Inferiors.html) ### Help Wanted [Old content no. 3](https://github.com/bitcoin/bitcoin/pull/15277#issuecomment-483318210) [Old content no. 2](https://github.com/bitcoin/bitcoin/pull/15277#issuecomment-471658439) <details> <summary>Old content no. 1</summary> As of now, the command described above to perform a build of bitcoin a lot like Gitian works, but fails at the `check-symbols` stage. This is because a few dynamic libraries are linked in that shouldn't be. Here's what `ldd src/bitcoind` looks like when built in a Guix container: ``` linux-vdso.so.1 (0x00007ffcc2d90000) libdl.so.2 => /gnu/store/h90vnqw0nwd0hhm1l5dgxsdrigddfmq4-glibc-2.28/lib/libdl.so.2 (0x00007fb7eda09000) librt.so.1 => /gnu/store/h90vnqw0nwd0hhm1l5dgxsdrigddfmq4-glibc-2.28/lib/librt.so.1 (0x00007fb7ed9ff000) libstdc++.so.6 => /gnu/store/4sqps8dczv3g7rwbdibfz6rf5jlk7w90-gcc-5.5.0-lib/lib/libstdc++.so.6 (0x00007fb7ed87c000) libpthread.so.0 => /gnu/store/h90vnqw0nwd0hhm1l5dgxsdrigddfmq4-glibc-2.28/lib/libpthread.so.0 (0x00007fb7ed85b000) libm.so.6 => /gnu/store/h90vnqw0nwd0hhm1l5dgxsdrigddfmq4-glibc-2.28/lib/libm.so.6 (0x00007fb7ed6da000) libgcc_s.so.1 => /gnu/store/4sqps8dczv3g7rwbdibfz6rf5jlk7w90-gcc-5.5.0-lib/lib/libgcc_s.so.1 (0x00007fb7ed6bf000) libc.so.6 => /gnu/store/h90vnqw0nwd0hhm1l5dgxsdrigddfmq4-glibc-2.28/lib/libc.so.6 (0x00007fb7ed506000) /gnu/store/h90vnqw0nwd0hhm1l5dgxsdrigddfmq4-glibc-2.28/lib/ld-linux-x86-64.so.2 => /usr/lib64/ld-linux-x86-64.so.2 (0x00007fb7ee3a0000) ``` And here's what it looks in one of our releases: ``` linux-vdso.so.1 (0x00007ffff52cd000) libpthread.so.0 => /usr/lib/libpthread.so.0 (0x00007f87726b4000) librt.so.1 => /usr/lib/librt.so.1 (0x00007f87726aa000) libm.so.6 => /usr/lib/libm.so.6 (0x00007f8772525000) libgcc_s.so.1 => /usr/lib/libgcc_s.so.1 (0x00007f877250b000) libc.so.6 => /usr/lib/libc.so.6 (0x00007f8772347000) /lib64/ld-linux-x86-64.so.2 => /usr/lib64/ld-linux-x86-64.so.2 (0x00007f8773392000) ``` ~~I suspect it is because my script does not apply the gitian-input patches [described in the release process](https://github.com/bitcoin/bitcoin/blob/master/doc/release-process.md#fetch-and-create-inputs-first-time-or-when-dependency-versions-change) but there is no description as to how these patches are applied.~~ It might also be something else entirely. Edit: It is something else. It appears that the gitian inputs are only used by [`gitian-win-signer.yml`](https://github.com/bitcoin/bitcoin/blob/d6e700e40f861ddd6743f4d13f0d6f6bc19093c2/contrib/gitian-descriptors/gitian-win-signer.yml#L14) </details> ### How to Help 1. Install Guix on your distro either [from source](https://www.gnu.org/software/guix/manual/en/html_node/Requirements.html) or perform a [binary installation](https://www.gnu.org/software/guix/manual/en/html_node/Binary-Installation.html#Binary-Installation) 2. Try out my branch and the command described above! ACKs for top commit: MarcoFalke: Thanks for the replies. ACK 751549b52a9a4cd27389d807ae67f02bbb39cd7f laanwj: ACK 751549b52a9a4cd27389d807ae67f02bbb39cd7f Tree-SHA512: 50e6ab58c6bda9a67125b6271daf7eff0ca57d0efa8941ed3cd951e5bf78b31552fc5e537b1e1bcf2d3cc918c63adf19d685aa117a0f851024dc67e697890a8d
2019-07-12 19:23:30 +02:00
(gnu packages file)
(gnu packages gawk)
(gnu packages gcc)
(gnu packages gnome)
(gnu packages image)
(gnu packages imagemagick)
Merge #17595: guix: Enable building for `x86_64-w64-mingw32` target a35e3235891d35daa167116cc70340140e883f06 guix: Appease travis. (Carl Dong) 0b66d22da5f53640e22f05adf880782c613e6d0f guix: Use gcc-9 for mingw-w64 instead of 8 (Carl Dong) ba0b99bdd613ba7f17c6247ece3001e1b44759a3 guix: Don't set MINGW_HAS_SECURE_API CFLAG in depends (Carl Dong) 93439a71eda49fb69f1e82966a23a946733aa6fa guix: Bump to upstream commit with mingw-w64 changes (Carl Dong) 35a96792dda9e78165b1598aeac7b2ab759e7be5 guix: Check mingw symbols, improve SSP fix docs (Carl Dong) 449d8fe25bbe25daacfc67aa89ca32b0a3254c5a guix: Expand on INT trap message (Carl Dong) 3f1f03c67a8e9edf487f08d272adb18b0a3942c8 guix: Spelling fixes (Carl Dong) ff821dd2a1c600488d11e7d9a20e9179ecc9144b guix: Reinstate make-ssp-fixed-gcc (Carl Dong) 360a9e0ad50a36ec79a1a160dbed3966689fd41c guix: Bump time-machine for mingw-w64 patches (Carl Dong) 93e41b7e3b54c17fd1b4c61ee95fc0dc2827e954 guix: Use gcc-8 for mingw-w64 instead of 7 (Carl Dong) ef4f7e4c45c60a69406134122f091c77c6ef740f guix: Set the well-known timezone env var (Carl Dong) acf4b3b3b5accf60a19441a0298ef27001b78e72 guix: Make x86_64-w64-mingw32 builds reproducible (Carl Dong) c4cce00eac691625b78b92f7dba0b7f57def19e5 guix: Remove dead links from README. (Carl Dong) df953a4c9a6143f45864757b706c88b6fa70545a guix: Appease shellcheck. (Carl Dong) 91897c95e191d293eb27d8af15cbeafc5b8f3895 guix: Improve guix-build.sh documentation (Carl Dong) 570d769c6c59b9f6d1a2b95b2ed60432cb33b3ba guix: Build support for Windows (Carl Dong) Pull request description: ~~Based on: https://github.com/bitcoin/bitcoin/pull/16519~~ Based on: #17933 (Time Machines are... shall we say... superior :grin:) This PR allows us to perform Guix builds for the `x86_64-w64-mingw32` target. We do this _without_ splitting up the build script like we do in Gitian by using this newfangled alien technology called `case` statements. (This is WIP and might be changed to `if` statements soon) ACKs for top commit: fanquake: ACK a35e3235891d35daa167116cc70340140e883f06 2/3 Tree-SHA512: c471951c23eb2cda919a71285d8b8f2580cb20f09d5db17b53e13dbd8813e01b3e7a83ea848e4913fd0f2bc12c6c133c5f76b54e65c0d89fed4dfd2e0be19875
2020-04-12 12:08:46 +02:00
(gnu packages installers)
Merge #15277: contrib: Enable building in Guix containers 751549b52a9a4cd27389d807ae67f02bbb39cd7f contrib: guix: Additional clarifications re: substitutes (Carl Dong) cd3e947f50db7cfe05c05b368c25742193729a62 contrib: guix: Various improvements. (Carl Dong) 8dff3e48a9e03299468ed3b342642f01f70da9db contrib: guix: Clarify SOURCE_DATE_EPOCH. (Carl Dong) 3e80ec3ea9691c7c89173de922a113e643fe976b contrib: Add deterministic Guix builds. (Carl Dong) Pull request description: ~~**This post is kept updated as this project progresses. Use this [latest update link](https://github.com/bitcoin/bitcoin/pull/15277#issuecomment-497303718) to see what's new.**~~ Please read the `README.md`. ----- ### Guix Introduction This PR enables building bitcoin in Guix containers. [Guix](https://www.gnu.org/software/guix/manual/en/html_node/Features.html) is a transactional package manager much like Nix, but unlike Nix, it has more of a focus on [bootstrappability](https://www.gnu.org/software/guix/manual/en/html_node/Bootstrapping.html) and [reproducibility](https://www.gnu.org/software/guix/blog/tags/reproducible-builds/) which are attractive for security-sensitive projects like bitcoin. ### Guix Build Walkthrough Please read the `README.md`. [Old instructions no. 4](https://github.com/bitcoin/bitcoin/pull/15277#issuecomment-497303718) [Old instructions no. 3](https://github.com/bitcoin/bitcoin/pull/15277#issuecomment-493827011) [Old instructions no. 2](https://github.com/bitcoin/bitcoin/pull/15277#issuecomment-471658439) <details> <summary>Old instructions no. 1</summary> In this PR, we define a Guix [manifest](https://www.gnu.org/software/guix/manual/en/html_node/Invoking-guix-package.html#profile_002dmanifest) in `contrib/guix/manifest.scm`, which declares what packages we want in our environment. We can then invoke ``` guix environment --manifest=contrib/guix/manifest.scm --container --pure --no-grafts --no-substitutes ``` To have Guix: 1. Build an environment containing the packages we defined in our `contrib/guix/manifest.scm` manifest from the Guix bootstrap binaries (see [bootstrappability](https://www.gnu.org/software/guix/manual/en/html_node/Bootstrapping.html) for more details). 2. Start a container with that environment that has no network access, and no access to the host's filesystem except to the `pwd` that it was started in. 3. Drop you into a shell in that container. > Note: if you don't want to wait hours for Guix to build the entire world from scratch, you can eliminate the `--no-substitutes` option to have Guix download from available binary sources. Note that this convenience doesn't necessarily compromise your security, as you can check that a package was built correctly after the fact using `guix build --check <packagename>` Therefore, we can perform a build of bitcoin much like in Gitian by invoking the following: ``` make -C depends -j"$(nproc)" download && \ cat contrib/guix/build.sh | guix environment --manifest=contrib/guix/manifest.scm --container --pure --no-grafts --no-substitutes ``` We don't include `make -C depends -j"$(nproc)" download` inside `contrib/guix/build.sh` because `contrib/guix/build.sh` is run inside the container, which has no network access (which is a good thing). </details> ### Rationale I believe that this represents a substantial improvement for the "supply chain security" of bitcoin because: 1. We no longer have to rely on Ubuntu for our build environment for our releases ([oh the horror](https://github.com/bitcoin/bitcoin/blob/72bd4ab867e3be0d8410403d9641c08288d343e3/contrib/gitian-descriptors/gitian-linux.yml#L10)), because Guix builds everything about the container, we can perform this on almost any Linux distro/system. 2. It is now much easier to determine what trusted binaries are in our supply chain, and even make a nice visualization! (see [bootstrappability](https://www.gnu.org/software/guix/manual/en/html_node/Bootstrapping.html)). 3. There is active effort among Guix folks to minimize the number of trusted binaries even further. OriansJ's [stage0](https://github.com/oriansj/stage0), and janneke's [Mes](https://www.gnu.org/software/mes/) all aim to achieve [reduced binary boostrap](http://joyofsource.com/reduced-binary-seed-bootstrap.html) for Guix. In fact, I believe if OriansJ gets his way, we will end up some day with only a single trusted binary: hex0 (a ~500 byte self-hosting hex assembler). ### Steps to Completion - [x] Successfully build bitcoin inside the Guix environment - [x] Make `check-symbols` pass - [x] Do the above but without nasty hacks - [x] Solve some of the more innocuous hacks - [ ] Make it cross-compile (HELP WANTED HERE) - [x] Linux - [x] x86_64-linux-gnu - [x] i686-linux-gnu - [x] aarch64-linux-gnu - [x] arm-linux-gnueabihf - [x] riscv64-linux-gnu - [ ] OS X - [ ] x86_64-apple-darwin14 - [ ] Windows - [ ] x86_64-w64-mingw32 - [ ] Maybe make importer for depends syntax - [ ] Document build process for future releases - [ ] Extra: Pin the revision of Guix that we build with with Guix [inferiors](https://www.gnu.org/software/guix/manual/en/html_node/Inferiors.html) ### Help Wanted [Old content no. 3](https://github.com/bitcoin/bitcoin/pull/15277#issuecomment-483318210) [Old content no. 2](https://github.com/bitcoin/bitcoin/pull/15277#issuecomment-471658439) <details> <summary>Old content no. 1</summary> As of now, the command described above to perform a build of bitcoin a lot like Gitian works, but fails at the `check-symbols` stage. This is because a few dynamic libraries are linked in that shouldn't be. Here's what `ldd src/bitcoind` looks like when built in a Guix container: ``` linux-vdso.so.1 (0x00007ffcc2d90000) libdl.so.2 => /gnu/store/h90vnqw0nwd0hhm1l5dgxsdrigddfmq4-glibc-2.28/lib/libdl.so.2 (0x00007fb7eda09000) librt.so.1 => /gnu/store/h90vnqw0nwd0hhm1l5dgxsdrigddfmq4-glibc-2.28/lib/librt.so.1 (0x00007fb7ed9ff000) libstdc++.so.6 => /gnu/store/4sqps8dczv3g7rwbdibfz6rf5jlk7w90-gcc-5.5.0-lib/lib/libstdc++.so.6 (0x00007fb7ed87c000) libpthread.so.0 => /gnu/store/h90vnqw0nwd0hhm1l5dgxsdrigddfmq4-glibc-2.28/lib/libpthread.so.0 (0x00007fb7ed85b000) libm.so.6 => /gnu/store/h90vnqw0nwd0hhm1l5dgxsdrigddfmq4-glibc-2.28/lib/libm.so.6 (0x00007fb7ed6da000) libgcc_s.so.1 => /gnu/store/4sqps8dczv3g7rwbdibfz6rf5jlk7w90-gcc-5.5.0-lib/lib/libgcc_s.so.1 (0x00007fb7ed6bf000) libc.so.6 => /gnu/store/h90vnqw0nwd0hhm1l5dgxsdrigddfmq4-glibc-2.28/lib/libc.so.6 (0x00007fb7ed506000) /gnu/store/h90vnqw0nwd0hhm1l5dgxsdrigddfmq4-glibc-2.28/lib/ld-linux-x86-64.so.2 => /usr/lib64/ld-linux-x86-64.so.2 (0x00007fb7ee3a0000) ``` And here's what it looks in one of our releases: ``` linux-vdso.so.1 (0x00007ffff52cd000) libpthread.so.0 => /usr/lib/libpthread.so.0 (0x00007f87726b4000) librt.so.1 => /usr/lib/librt.so.1 (0x00007f87726aa000) libm.so.6 => /usr/lib/libm.so.6 (0x00007f8772525000) libgcc_s.so.1 => /usr/lib/libgcc_s.so.1 (0x00007f877250b000) libc.so.6 => /usr/lib/libc.so.6 (0x00007f8772347000) /lib64/ld-linux-x86-64.so.2 => /usr/lib64/ld-linux-x86-64.so.2 (0x00007f8773392000) ``` ~~I suspect it is because my script does not apply the gitian-input patches [described in the release process](https://github.com/bitcoin/bitcoin/blob/master/doc/release-process.md#fetch-and-create-inputs-first-time-or-when-dependency-versions-change) but there is no description as to how these patches are applied.~~ It might also be something else entirely. Edit: It is something else. It appears that the gitian inputs are only used by [`gitian-win-signer.yml`](https://github.com/bitcoin/bitcoin/blob/d6e700e40f861ddd6743f4d13f0d6f6bc19093c2/contrib/gitian-descriptors/gitian-win-signer.yml#L14) </details> ### How to Help 1. Install Guix on your distro either [from source](https://www.gnu.org/software/guix/manual/en/html_node/Requirements.html) or perform a [binary installation](https://www.gnu.org/software/guix/manual/en/html_node/Binary-Installation.html#Binary-Installation) 2. Try out my branch and the command described above! ACKs for top commit: MarcoFalke: Thanks for the replies. ACK 751549b52a9a4cd27389d807ae67f02bbb39cd7f laanwj: ACK 751549b52a9a4cd27389d807ae67f02bbb39cd7f Tree-SHA512: 50e6ab58c6bda9a67125b6271daf7eff0ca57d0efa8941ed3cd951e5bf78b31552fc5e537b1e1bcf2d3cc918c63adf19d685aa117a0f851024dc67e697890a8d
2019-07-12 19:23:30 +02:00
(gnu packages linux)
(gnu packages llvm)
Merge #17595: guix: Enable building for `x86_64-w64-mingw32` target a35e3235891d35daa167116cc70340140e883f06 guix: Appease travis. (Carl Dong) 0b66d22da5f53640e22f05adf880782c613e6d0f guix: Use gcc-9 for mingw-w64 instead of 8 (Carl Dong) ba0b99bdd613ba7f17c6247ece3001e1b44759a3 guix: Don't set MINGW_HAS_SECURE_API CFLAG in depends (Carl Dong) 93439a71eda49fb69f1e82966a23a946733aa6fa guix: Bump to upstream commit with mingw-w64 changes (Carl Dong) 35a96792dda9e78165b1598aeac7b2ab759e7be5 guix: Check mingw symbols, improve SSP fix docs (Carl Dong) 449d8fe25bbe25daacfc67aa89ca32b0a3254c5a guix: Expand on INT trap message (Carl Dong) 3f1f03c67a8e9edf487f08d272adb18b0a3942c8 guix: Spelling fixes (Carl Dong) ff821dd2a1c600488d11e7d9a20e9179ecc9144b guix: Reinstate make-ssp-fixed-gcc (Carl Dong) 360a9e0ad50a36ec79a1a160dbed3966689fd41c guix: Bump time-machine for mingw-w64 patches (Carl Dong) 93e41b7e3b54c17fd1b4c61ee95fc0dc2827e954 guix: Use gcc-8 for mingw-w64 instead of 7 (Carl Dong) ef4f7e4c45c60a69406134122f091c77c6ef740f guix: Set the well-known timezone env var (Carl Dong) acf4b3b3b5accf60a19441a0298ef27001b78e72 guix: Make x86_64-w64-mingw32 builds reproducible (Carl Dong) c4cce00eac691625b78b92f7dba0b7f57def19e5 guix: Remove dead links from README. (Carl Dong) df953a4c9a6143f45864757b706c88b6fa70545a guix: Appease shellcheck. (Carl Dong) 91897c95e191d293eb27d8af15cbeafc5b8f3895 guix: Improve guix-build.sh documentation (Carl Dong) 570d769c6c59b9f6d1a2b95b2ed60432cb33b3ba guix: Build support for Windows (Carl Dong) Pull request description: ~~Based on: https://github.com/bitcoin/bitcoin/pull/16519~~ Based on: #17933 (Time Machines are... shall we say... superior :grin:) This PR allows us to perform Guix builds for the `x86_64-w64-mingw32` target. We do this _without_ splitting up the build script like we do in Gitian by using this newfangled alien technology called `case` statements. (This is WIP and might be changed to `if` statements soon) ACKs for top commit: fanquake: ACK a35e3235891d35daa167116cc70340140e883f06 2/3 Tree-SHA512: c471951c23eb2cda919a71285d8b8f2580cb20f09d5db17b53e13dbd8813e01b3e7a83ea848e4913fd0f2bc12c6c133c5f76b54e65c0d89fed4dfd2e0be19875
2020-04-12 12:08:46 +02:00
(gnu packages mingw)
Merge bitcoin/bitcoin#22182: guix: Overhaul how guix-{attest,verify} works and hierarchy e2c40a4ed5272d72fea997bd936fba28bb753226 guix-attest: Error out if SHA256SUMS is unexpected (Carl Dong) 4cc35daed557f38b080360a89036b2e97a6f78c2 Rewrite guix-{attest,verify} for new hier (Carl Dong) 28a9c9b83924f585b397f0f3b8e9e73780ac0ad6 Make SHA256SUMS fragment right after build (Carl Dong) Pull request description: Based on: #22075 Code reviewers: I recommend reading the new `guix-{attest,verify}` files instead of trying to read the diff The following changes resolve many usability improvements which were pointed out to me: 1. Some maintainers like to extract their "uncodesigned tarball" inside the `output/` directory, resulting in the older `guix-attest` mistakenly attesting to the extracted contents 2. Maintainers whose GPG keys reside on an external smartcard often need to physically interact with the smartcard as a way to approve the signing operation, having one signature per platform means a lot of fidgeting 3. Maintainers wishing to sign on a separate machine now has the option of transferring only a subtree of `output/`, namely `output/*/SHA256SUMS.part`, in order to perform a signature (you may need to specify an `$OUTDIR_BASE` env var) 4. An `all.SHA256SUMS` file should be usable as the base `SHA256SUMS` in bitcoin core torrents and on the release server. For those who sign on an separate machine than the one you do builds on, the following steps will work: 1. `env GUIX_SIGS_REPO=/home/achow101/guix.sigs SIGNER=achow101 NO_SIGN=1 ./contrib/guix/guix-attest` 2. Copy `/home/achow101/guix.sigs/<tag>/achow101` (which does not yet have signatures) to signing machine 3. Sign the `SHA256SUMS` files: ```bash for i in "<path-to-achow101>/*.SHA256SUMS"; do gpg --detach-sign --local-user "<your-key-here>" --armor --output "$i"{.asc,} done ``` 5. Upload `<path-to-achow101>` (now with signatures) to `guix.sigs` ----- After this change, output directories will now include a `SHA256SUMS.part` fragment, created immediately after a successful build: ``` output └── x86_64-w64-mingw32 ├── bitcoin-4e069f7589da-win64-debug.zip ├── bitcoin-4e069f7589da-win64-setup-unsigned.exe ├── bitcoin-4e069f7589da-win64.zip ├── bitcoin-4e069f7589da-win-unsigned.tar.gz └── SHA256SUMS.part ``` These `SHA256SUMS.part` fragments look something like: ``` 3ebd7262b1a0a5bb757fef1f70e7e14033c70f98c059bc4dbfee5d1992b25825 dist-archive/bitcoin-4e069f7589da.tar.gz def2e7d3de5ab3e3f955344e75151df4f33713f9101f5295bd13c9375bdf633b x86_64-w64-mingw32/bitcoin-4e069f7589da-win64-debug.zip 643049fe3ee4a4e83a1739607e67b11b7c9b1a66208a6f35a9ff634ba795500e x86_64-w64-mingw32/bitcoin-4e069f7589da-win64-setup-unsigned.exe a247a1ccec0ccc2e138c648284bd01f6a761f2d8d6d07d91b5b4a6670ec3f288 x86_64-w64-mingw32/bitcoin-4e069f7589da-win-unsigned.tar.gz fab76a836dcc592e39c04fd2396696633fb6eb56e39ecbf6c909bd173ed4280c x86_64-w64-mingw32/bitcoin-4e069f7589da-win64.zip ``` Meaning that they are valid `SHA256SUMS` files when `sha256sum --check`'d at the `guix-build-*/output` directory level When `guix-attest` is invoked, these `SHA256SUMS.part` files are combined and sorted (by `-k2`, `LC_ALL=C`) to create: 1. `noncodesigned.SHA256SUMS` for a manifest of all non-codesigned outputs, and 3. `all.SHA256SUMS` for a manifest of all outputs including non-codesigned outputs Then both files are signed, resulting in the following `guix.sigs` hierarchy: ``` 4e069f7589da/ └── dongcarl ├── all.SHA256SUMS ├── all.SHA256SUMS.asc ├── noncodesigned.SHA256SUMS └── noncodesigned.SHA256SUMS.asc ``` ACKs for top commit: achow101: ACK e2c40a4ed5272d72fea997bd936fba28bb753226 hebasto: ACK e2c40a4ed5272d72fea997bd936fba28bb753226, tested on Linux Mint 20.1 (x86_64) with and w/o `NO_SIGN=1`. Changes in `contrib/guix/libexec/codesign.sh` and `contrib/guix/guix-verify` are reviewed only. Tree-SHA512: 618aacefb0eb6595735a9ab6a98ea6598fce65f9ccf33fa1e7ef93bf140c0f6cfc16e34870c6aa3e4777dd3f004b92a82a994141879870141742df948ec59c1f
2021-06-17 07:10:29 +02:00
(gnu packages moreutils)
Merge #15277: contrib: Enable building in Guix containers 751549b52a9a4cd27389d807ae67f02bbb39cd7f contrib: guix: Additional clarifications re: substitutes (Carl Dong) cd3e947f50db7cfe05c05b368c25742193729a62 contrib: guix: Various improvements. (Carl Dong) 8dff3e48a9e03299468ed3b342642f01f70da9db contrib: guix: Clarify SOURCE_DATE_EPOCH. (Carl Dong) 3e80ec3ea9691c7c89173de922a113e643fe976b contrib: Add deterministic Guix builds. (Carl Dong) Pull request description: ~~**This post is kept updated as this project progresses. Use this [latest update link](https://github.com/bitcoin/bitcoin/pull/15277#issuecomment-497303718) to see what's new.**~~ Please read the `README.md`. ----- ### Guix Introduction This PR enables building bitcoin in Guix containers. [Guix](https://www.gnu.org/software/guix/manual/en/html_node/Features.html) is a transactional package manager much like Nix, but unlike Nix, it has more of a focus on [bootstrappability](https://www.gnu.org/software/guix/manual/en/html_node/Bootstrapping.html) and [reproducibility](https://www.gnu.org/software/guix/blog/tags/reproducible-builds/) which are attractive for security-sensitive projects like bitcoin. ### Guix Build Walkthrough Please read the `README.md`. [Old instructions no. 4](https://github.com/bitcoin/bitcoin/pull/15277#issuecomment-497303718) [Old instructions no. 3](https://github.com/bitcoin/bitcoin/pull/15277#issuecomment-493827011) [Old instructions no. 2](https://github.com/bitcoin/bitcoin/pull/15277#issuecomment-471658439) <details> <summary>Old instructions no. 1</summary> In this PR, we define a Guix [manifest](https://www.gnu.org/software/guix/manual/en/html_node/Invoking-guix-package.html#profile_002dmanifest) in `contrib/guix/manifest.scm`, which declares what packages we want in our environment. We can then invoke ``` guix environment --manifest=contrib/guix/manifest.scm --container --pure --no-grafts --no-substitutes ``` To have Guix: 1. Build an environment containing the packages we defined in our `contrib/guix/manifest.scm` manifest from the Guix bootstrap binaries (see [bootstrappability](https://www.gnu.org/software/guix/manual/en/html_node/Bootstrapping.html) for more details). 2. Start a container with that environment that has no network access, and no access to the host's filesystem except to the `pwd` that it was started in. 3. Drop you into a shell in that container. > Note: if you don't want to wait hours for Guix to build the entire world from scratch, you can eliminate the `--no-substitutes` option to have Guix download from available binary sources. Note that this convenience doesn't necessarily compromise your security, as you can check that a package was built correctly after the fact using `guix build --check <packagename>` Therefore, we can perform a build of bitcoin much like in Gitian by invoking the following: ``` make -C depends -j"$(nproc)" download && \ cat contrib/guix/build.sh | guix environment --manifest=contrib/guix/manifest.scm --container --pure --no-grafts --no-substitutes ``` We don't include `make -C depends -j"$(nproc)" download` inside `contrib/guix/build.sh` because `contrib/guix/build.sh` is run inside the container, which has no network access (which is a good thing). </details> ### Rationale I believe that this represents a substantial improvement for the "supply chain security" of bitcoin because: 1. We no longer have to rely on Ubuntu for our build environment for our releases ([oh the horror](https://github.com/bitcoin/bitcoin/blob/72bd4ab867e3be0d8410403d9641c08288d343e3/contrib/gitian-descriptors/gitian-linux.yml#L10)), because Guix builds everything about the container, we can perform this on almost any Linux distro/system. 2. It is now much easier to determine what trusted binaries are in our supply chain, and even make a nice visualization! (see [bootstrappability](https://www.gnu.org/software/guix/manual/en/html_node/Bootstrapping.html)). 3. There is active effort among Guix folks to minimize the number of trusted binaries even further. OriansJ's [stage0](https://github.com/oriansj/stage0), and janneke's [Mes](https://www.gnu.org/software/mes/) all aim to achieve [reduced binary boostrap](http://joyofsource.com/reduced-binary-seed-bootstrap.html) for Guix. In fact, I believe if OriansJ gets his way, we will end up some day with only a single trusted binary: hex0 (a ~500 byte self-hosting hex assembler). ### Steps to Completion - [x] Successfully build bitcoin inside the Guix environment - [x] Make `check-symbols` pass - [x] Do the above but without nasty hacks - [x] Solve some of the more innocuous hacks - [ ] Make it cross-compile (HELP WANTED HERE) - [x] Linux - [x] x86_64-linux-gnu - [x] i686-linux-gnu - [x] aarch64-linux-gnu - [x] arm-linux-gnueabihf - [x] riscv64-linux-gnu - [ ] OS X - [ ] x86_64-apple-darwin14 - [ ] Windows - [ ] x86_64-w64-mingw32 - [ ] Maybe make importer for depends syntax - [ ] Document build process for future releases - [ ] Extra: Pin the revision of Guix that we build with with Guix [inferiors](https://www.gnu.org/software/guix/manual/en/html_node/Inferiors.html) ### Help Wanted [Old content no. 3](https://github.com/bitcoin/bitcoin/pull/15277#issuecomment-483318210) [Old content no. 2](https://github.com/bitcoin/bitcoin/pull/15277#issuecomment-471658439) <details> <summary>Old content no. 1</summary> As of now, the command described above to perform a build of bitcoin a lot like Gitian works, but fails at the `check-symbols` stage. This is because a few dynamic libraries are linked in that shouldn't be. Here's what `ldd src/bitcoind` looks like when built in a Guix container: ``` linux-vdso.so.1 (0x00007ffcc2d90000) libdl.so.2 => /gnu/store/h90vnqw0nwd0hhm1l5dgxsdrigddfmq4-glibc-2.28/lib/libdl.so.2 (0x00007fb7eda09000) librt.so.1 => /gnu/store/h90vnqw0nwd0hhm1l5dgxsdrigddfmq4-glibc-2.28/lib/librt.so.1 (0x00007fb7ed9ff000) libstdc++.so.6 => /gnu/store/4sqps8dczv3g7rwbdibfz6rf5jlk7w90-gcc-5.5.0-lib/lib/libstdc++.so.6 (0x00007fb7ed87c000) libpthread.so.0 => /gnu/store/h90vnqw0nwd0hhm1l5dgxsdrigddfmq4-glibc-2.28/lib/libpthread.so.0 (0x00007fb7ed85b000) libm.so.6 => /gnu/store/h90vnqw0nwd0hhm1l5dgxsdrigddfmq4-glibc-2.28/lib/libm.so.6 (0x00007fb7ed6da000) libgcc_s.so.1 => /gnu/store/4sqps8dczv3g7rwbdibfz6rf5jlk7w90-gcc-5.5.0-lib/lib/libgcc_s.so.1 (0x00007fb7ed6bf000) libc.so.6 => /gnu/store/h90vnqw0nwd0hhm1l5dgxsdrigddfmq4-glibc-2.28/lib/libc.so.6 (0x00007fb7ed506000) /gnu/store/h90vnqw0nwd0hhm1l5dgxsdrigddfmq4-glibc-2.28/lib/ld-linux-x86-64.so.2 => /usr/lib64/ld-linux-x86-64.so.2 (0x00007fb7ee3a0000) ``` And here's what it looks in one of our releases: ``` linux-vdso.so.1 (0x00007ffff52cd000) libpthread.so.0 => /usr/lib/libpthread.so.0 (0x00007f87726b4000) librt.so.1 => /usr/lib/librt.so.1 (0x00007f87726aa000) libm.so.6 => /usr/lib/libm.so.6 (0x00007f8772525000) libgcc_s.so.1 => /usr/lib/libgcc_s.so.1 (0x00007f877250b000) libc.so.6 => /usr/lib/libc.so.6 (0x00007f8772347000) /lib64/ld-linux-x86-64.so.2 => /usr/lib64/ld-linux-x86-64.so.2 (0x00007f8773392000) ``` ~~I suspect it is because my script does not apply the gitian-input patches [described in the release process](https://github.com/bitcoin/bitcoin/blob/master/doc/release-process.md#fetch-and-create-inputs-first-time-or-when-dependency-versions-change) but there is no description as to how these patches are applied.~~ It might also be something else entirely. Edit: It is something else. It appears that the gitian inputs are only used by [`gitian-win-signer.yml`](https://github.com/bitcoin/bitcoin/blob/d6e700e40f861ddd6743f4d13f0d6f6bc19093c2/contrib/gitian-descriptors/gitian-win-signer.yml#L14) </details> ### How to Help 1. Install Guix on your distro either [from source](https://www.gnu.org/software/guix/manual/en/html_node/Requirements.html) or perform a [binary installation](https://www.gnu.org/software/guix/manual/en/html_node/Binary-Installation.html#Binary-Installation) 2. Try out my branch and the command described above! ACKs for top commit: MarcoFalke: Thanks for the replies. ACK 751549b52a9a4cd27389d807ae67f02bbb39cd7f laanwj: ACK 751549b52a9a4cd27389d807ae67f02bbb39cd7f Tree-SHA512: 50e6ab58c6bda9a67125b6271daf7eff0ca57d0efa8941ed3cd951e5bf78b31552fc5e537b1e1bcf2d3cc918c63adf19d685aa117a0f851024dc67e697890a8d
2019-07-12 19:23:30 +02:00
(gnu packages perl)
(gnu packages pkg-config)
(gnu packages python)
Merge bitcoin/bitcoin#21239: guix: Add codesignature attachment support for osx+win ee883201cf134952284632e9e9ae72bf1c8c792f guix: repro: Sort find output in libtool for gcc-8 (Carl Dong) ee0a67c32a8861eab650bf8894af06807578eba0 codesigning: Use SHA256 as digest for osslsigncode (Windows) (Carl Dong) 38eb91eb0616ed6dbe34c23e11588d130fef07f8 guix: Add codesigning functionality (Carl Dong) bac2690e6f683fcedb883fe1d32f3c33c628a141 guix: Package codesigning tools (Carl Dong) 0a2176d47767972e4cd5ed302c1dbeedece1708b guix: Reindent existing manifest.scm (Carl Dong) c090a3e9238ba2df07875b4708e908d8dca4ed9b Makefile.am: use APP_DIST_DIR instead of hard-coding dist (Carl Dong) Pull request description: This is the last PR before we reach feature-parity with the Gitian process! Note: I tried using the `Makefile` inside the distsrc to make the dmg instead of manually listing out the commands, but `make` seems to want to re-make a lot of other files which broke the dmg. The workflow looks something like this: 1. `env [ FOO=bar... ] ./contrib/guix/guix-build` (add additional env vars as necessary) 2. Codesigners only: 1. Copy `guix-build-<short-id>/output/x86_64-apple-darwin18/bitcoin-<short-id>-osx-unsigned.tar.gz` and `guix-build-<short-id>/output/x86_64-w64-mingw32/bitcoin-<short-id>-win-unsigned.tar.gz` to signing computer 2. Codesign with `./detached-sig-create.sh` inside the tarball 3. Upload contents of `signature-{osx,win}.tar.gz` to https://github.com/bitcoin-core/bitcoin-detached-sigs (as a new tag) 3. Checkout new tag for `bitcoin-core/bitcoin-detached-sigs` with the detached signatures 4. `env [ FOO=bar... ] DETACHED_SIGS_REPO=<path/to/bitcoin-detached-sigs> ./contrib/guix/guix-codesign` (modify env vars as necessary) 5. Make sure `guix.sigs` is cloned and updated 6. `env GUIX_SIGS_REPO=<path/to/guix.sigs> SIGNER=0x96AB007F1A7ED999=dongcarl ./contrib/guix/guix-attest` (modify env vars as necessary) 7. Commit your new signatures and SHA256SUMS in `guix.sigs` 8. Optionally, after there are multiple signatures in `guix.sigs`: `env GUIX_SIGS_REPO=<path/to/guix.sigs> ./contrib/guix/guix-verify` ACKs for top commit: laanwj: Tested ACK ee883201cf134952284632e9e9ae72bf1c8c792f achow101: ACK ee883201cf134952284632e9e9ae72bf1c8c792f Tree-SHA512: e812a07a5f19f900600c70cb9c717769ef544a6c0c12760b5558b76b6b37df863257f3dbf38b0757e6e06e334470267e94c9f2bdbc27409d6837b1a0bfc6acbc
2021-05-24 15:31:05 +02:00
(gnu packages python-web)
Merge #15277: contrib: Enable building in Guix containers 751549b52a9a4cd27389d807ae67f02bbb39cd7f contrib: guix: Additional clarifications re: substitutes (Carl Dong) cd3e947f50db7cfe05c05b368c25742193729a62 contrib: guix: Various improvements. (Carl Dong) 8dff3e48a9e03299468ed3b342642f01f70da9db contrib: guix: Clarify SOURCE_DATE_EPOCH. (Carl Dong) 3e80ec3ea9691c7c89173de922a113e643fe976b contrib: Add deterministic Guix builds. (Carl Dong) Pull request description: ~~**This post is kept updated as this project progresses. Use this [latest update link](https://github.com/bitcoin/bitcoin/pull/15277#issuecomment-497303718) to see what's new.**~~ Please read the `README.md`. ----- ### Guix Introduction This PR enables building bitcoin in Guix containers. [Guix](https://www.gnu.org/software/guix/manual/en/html_node/Features.html) is a transactional package manager much like Nix, but unlike Nix, it has more of a focus on [bootstrappability](https://www.gnu.org/software/guix/manual/en/html_node/Bootstrapping.html) and [reproducibility](https://www.gnu.org/software/guix/blog/tags/reproducible-builds/) which are attractive for security-sensitive projects like bitcoin. ### Guix Build Walkthrough Please read the `README.md`. [Old instructions no. 4](https://github.com/bitcoin/bitcoin/pull/15277#issuecomment-497303718) [Old instructions no. 3](https://github.com/bitcoin/bitcoin/pull/15277#issuecomment-493827011) [Old instructions no. 2](https://github.com/bitcoin/bitcoin/pull/15277#issuecomment-471658439) <details> <summary>Old instructions no. 1</summary> In this PR, we define a Guix [manifest](https://www.gnu.org/software/guix/manual/en/html_node/Invoking-guix-package.html#profile_002dmanifest) in `contrib/guix/manifest.scm`, which declares what packages we want in our environment. We can then invoke ``` guix environment --manifest=contrib/guix/manifest.scm --container --pure --no-grafts --no-substitutes ``` To have Guix: 1. Build an environment containing the packages we defined in our `contrib/guix/manifest.scm` manifest from the Guix bootstrap binaries (see [bootstrappability](https://www.gnu.org/software/guix/manual/en/html_node/Bootstrapping.html) for more details). 2. Start a container with that environment that has no network access, and no access to the host's filesystem except to the `pwd` that it was started in. 3. Drop you into a shell in that container. > Note: if you don't want to wait hours for Guix to build the entire world from scratch, you can eliminate the `--no-substitutes` option to have Guix download from available binary sources. Note that this convenience doesn't necessarily compromise your security, as you can check that a package was built correctly after the fact using `guix build --check <packagename>` Therefore, we can perform a build of bitcoin much like in Gitian by invoking the following: ``` make -C depends -j"$(nproc)" download && \ cat contrib/guix/build.sh | guix environment --manifest=contrib/guix/manifest.scm --container --pure --no-grafts --no-substitutes ``` We don't include `make -C depends -j"$(nproc)" download` inside `contrib/guix/build.sh` because `contrib/guix/build.sh` is run inside the container, which has no network access (which is a good thing). </details> ### Rationale I believe that this represents a substantial improvement for the "supply chain security" of bitcoin because: 1. We no longer have to rely on Ubuntu for our build environment for our releases ([oh the horror](https://github.com/bitcoin/bitcoin/blob/72bd4ab867e3be0d8410403d9641c08288d343e3/contrib/gitian-descriptors/gitian-linux.yml#L10)), because Guix builds everything about the container, we can perform this on almost any Linux distro/system. 2. It is now much easier to determine what trusted binaries are in our supply chain, and even make a nice visualization! (see [bootstrappability](https://www.gnu.org/software/guix/manual/en/html_node/Bootstrapping.html)). 3. There is active effort among Guix folks to minimize the number of trusted binaries even further. OriansJ's [stage0](https://github.com/oriansj/stage0), and janneke's [Mes](https://www.gnu.org/software/mes/) all aim to achieve [reduced binary boostrap](http://joyofsource.com/reduced-binary-seed-bootstrap.html) for Guix. In fact, I believe if OriansJ gets his way, we will end up some day with only a single trusted binary: hex0 (a ~500 byte self-hosting hex assembler). ### Steps to Completion - [x] Successfully build bitcoin inside the Guix environment - [x] Make `check-symbols` pass - [x] Do the above but without nasty hacks - [x] Solve some of the more innocuous hacks - [ ] Make it cross-compile (HELP WANTED HERE) - [x] Linux - [x] x86_64-linux-gnu - [x] i686-linux-gnu - [x] aarch64-linux-gnu - [x] arm-linux-gnueabihf - [x] riscv64-linux-gnu - [ ] OS X - [ ] x86_64-apple-darwin14 - [ ] Windows - [ ] x86_64-w64-mingw32 - [ ] Maybe make importer for depends syntax - [ ] Document build process for future releases - [ ] Extra: Pin the revision of Guix that we build with with Guix [inferiors](https://www.gnu.org/software/guix/manual/en/html_node/Inferiors.html) ### Help Wanted [Old content no. 3](https://github.com/bitcoin/bitcoin/pull/15277#issuecomment-483318210) [Old content no. 2](https://github.com/bitcoin/bitcoin/pull/15277#issuecomment-471658439) <details> <summary>Old content no. 1</summary> As of now, the command described above to perform a build of bitcoin a lot like Gitian works, but fails at the `check-symbols` stage. This is because a few dynamic libraries are linked in that shouldn't be. Here's what `ldd src/bitcoind` looks like when built in a Guix container: ``` linux-vdso.so.1 (0x00007ffcc2d90000) libdl.so.2 => /gnu/store/h90vnqw0nwd0hhm1l5dgxsdrigddfmq4-glibc-2.28/lib/libdl.so.2 (0x00007fb7eda09000) librt.so.1 => /gnu/store/h90vnqw0nwd0hhm1l5dgxsdrigddfmq4-glibc-2.28/lib/librt.so.1 (0x00007fb7ed9ff000) libstdc++.so.6 => /gnu/store/4sqps8dczv3g7rwbdibfz6rf5jlk7w90-gcc-5.5.0-lib/lib/libstdc++.so.6 (0x00007fb7ed87c000) libpthread.so.0 => /gnu/store/h90vnqw0nwd0hhm1l5dgxsdrigddfmq4-glibc-2.28/lib/libpthread.so.0 (0x00007fb7ed85b000) libm.so.6 => /gnu/store/h90vnqw0nwd0hhm1l5dgxsdrigddfmq4-glibc-2.28/lib/libm.so.6 (0x00007fb7ed6da000) libgcc_s.so.1 => /gnu/store/4sqps8dczv3g7rwbdibfz6rf5jlk7w90-gcc-5.5.0-lib/lib/libgcc_s.so.1 (0x00007fb7ed6bf000) libc.so.6 => /gnu/store/h90vnqw0nwd0hhm1l5dgxsdrigddfmq4-glibc-2.28/lib/libc.so.6 (0x00007fb7ed506000) /gnu/store/h90vnqw0nwd0hhm1l5dgxsdrigddfmq4-glibc-2.28/lib/ld-linux-x86-64.so.2 => /usr/lib64/ld-linux-x86-64.so.2 (0x00007fb7ee3a0000) ``` And here's what it looks in one of our releases: ``` linux-vdso.so.1 (0x00007ffff52cd000) libpthread.so.0 => /usr/lib/libpthread.so.0 (0x00007f87726b4000) librt.so.1 => /usr/lib/librt.so.1 (0x00007f87726aa000) libm.so.6 => /usr/lib/libm.so.6 (0x00007f8772525000) libgcc_s.so.1 => /usr/lib/libgcc_s.so.1 (0x00007f877250b000) libc.so.6 => /usr/lib/libc.so.6 (0x00007f8772347000) /lib64/ld-linux-x86-64.so.2 => /usr/lib64/ld-linux-x86-64.so.2 (0x00007f8773392000) ``` ~~I suspect it is because my script does not apply the gitian-input patches [described in the release process](https://github.com/bitcoin/bitcoin/blob/master/doc/release-process.md#fetch-and-create-inputs-first-time-or-when-dependency-versions-change) but there is no description as to how these patches are applied.~~ It might also be something else entirely. Edit: It is something else. It appears that the gitian inputs are only used by [`gitian-win-signer.yml`](https://github.com/bitcoin/bitcoin/blob/d6e700e40f861ddd6743f4d13f0d6f6bc19093c2/contrib/gitian-descriptors/gitian-win-signer.yml#L14) </details> ### How to Help 1. Install Guix on your distro either [from source](https://www.gnu.org/software/guix/manual/en/html_node/Requirements.html) or perform a [binary installation](https://www.gnu.org/software/guix/manual/en/html_node/Binary-Installation.html#Binary-Installation) 2. Try out my branch and the command described above! ACKs for top commit: MarcoFalke: Thanks for the replies. ACK 751549b52a9a4cd27389d807ae67f02bbb39cd7f laanwj: ACK 751549b52a9a4cd27389d807ae67f02bbb39cd7f Tree-SHA512: 50e6ab58c6bda9a67125b6271daf7eff0ca57d0efa8941ed3cd951e5bf78b31552fc5e537b1e1bcf2d3cc918c63adf19d685aa117a0f851024dc67e697890a8d
2019-07-12 19:23:30 +02:00
(gnu packages shells)
(gnu packages bison)
Merge bitcoin/bitcoin#21239: guix: Add codesignature attachment support for osx+win ee883201cf134952284632e9e9ae72bf1c8c792f guix: repro: Sort find output in libtool for gcc-8 (Carl Dong) ee0a67c32a8861eab650bf8894af06807578eba0 codesigning: Use SHA256 as digest for osslsigncode (Windows) (Carl Dong) 38eb91eb0616ed6dbe34c23e11588d130fef07f8 guix: Add codesigning functionality (Carl Dong) bac2690e6f683fcedb883fe1d32f3c33c628a141 guix: Package codesigning tools (Carl Dong) 0a2176d47767972e4cd5ed302c1dbeedece1708b guix: Reindent existing manifest.scm (Carl Dong) c090a3e9238ba2df07875b4708e908d8dca4ed9b Makefile.am: use APP_DIST_DIR instead of hard-coding dist (Carl Dong) Pull request description: This is the last PR before we reach feature-parity with the Gitian process! Note: I tried using the `Makefile` inside the distsrc to make the dmg instead of manually listing out the commands, but `make` seems to want to re-make a lot of other files which broke the dmg. The workflow looks something like this: 1. `env [ FOO=bar... ] ./contrib/guix/guix-build` (add additional env vars as necessary) 2. Codesigners only: 1. Copy `guix-build-<short-id>/output/x86_64-apple-darwin18/bitcoin-<short-id>-osx-unsigned.tar.gz` and `guix-build-<short-id>/output/x86_64-w64-mingw32/bitcoin-<short-id>-win-unsigned.tar.gz` to signing computer 2. Codesign with `./detached-sig-create.sh` inside the tarball 3. Upload contents of `signature-{osx,win}.tar.gz` to https://github.com/bitcoin-core/bitcoin-detached-sigs (as a new tag) 3. Checkout new tag for `bitcoin-core/bitcoin-detached-sigs` with the detached signatures 4. `env [ FOO=bar... ] DETACHED_SIGS_REPO=<path/to/bitcoin-detached-sigs> ./contrib/guix/guix-codesign` (modify env vars as necessary) 5. Make sure `guix.sigs` is cloned and updated 6. `env GUIX_SIGS_REPO=<path/to/guix.sigs> SIGNER=0x96AB007F1A7ED999=dongcarl ./contrib/guix/guix-attest` (modify env vars as necessary) 7. Commit your new signatures and SHA256SUMS in `guix.sigs` 8. Optionally, after there are multiple signatures in `guix.sigs`: `env GUIX_SIGS_REPO=<path/to/guix.sigs> ./contrib/guix/guix-verify` ACKs for top commit: laanwj: Tested ACK ee883201cf134952284632e9e9ae72bf1c8c792f achow101: ACK ee883201cf134952284632e9e9ae72bf1c8c792f Tree-SHA512: e812a07a5f19f900600c70cb9c717769ef544a6c0c12760b5558b76b6b37df863257f3dbf38b0757e6e06e334470267e94c9f2bdbc27409d6837b1a0bfc6acbc
2021-05-24 15:31:05 +02:00
(gnu packages tls)
(gnu packages version-control)
(guix build-system font)
Merge #17595: guix: Enable building for `x86_64-w64-mingw32` target a35e3235891d35daa167116cc70340140e883f06 guix: Appease travis. (Carl Dong) 0b66d22da5f53640e22f05adf880782c613e6d0f guix: Use gcc-9 for mingw-w64 instead of 8 (Carl Dong) ba0b99bdd613ba7f17c6247ece3001e1b44759a3 guix: Don't set MINGW_HAS_SECURE_API CFLAG in depends (Carl Dong) 93439a71eda49fb69f1e82966a23a946733aa6fa guix: Bump to upstream commit with mingw-w64 changes (Carl Dong) 35a96792dda9e78165b1598aeac7b2ab759e7be5 guix: Check mingw symbols, improve SSP fix docs (Carl Dong) 449d8fe25bbe25daacfc67aa89ca32b0a3254c5a guix: Expand on INT trap message (Carl Dong) 3f1f03c67a8e9edf487f08d272adb18b0a3942c8 guix: Spelling fixes (Carl Dong) ff821dd2a1c600488d11e7d9a20e9179ecc9144b guix: Reinstate make-ssp-fixed-gcc (Carl Dong) 360a9e0ad50a36ec79a1a160dbed3966689fd41c guix: Bump time-machine for mingw-w64 patches (Carl Dong) 93e41b7e3b54c17fd1b4c61ee95fc0dc2827e954 guix: Use gcc-8 for mingw-w64 instead of 7 (Carl Dong) ef4f7e4c45c60a69406134122f091c77c6ef740f guix: Set the well-known timezone env var (Carl Dong) acf4b3b3b5accf60a19441a0298ef27001b78e72 guix: Make x86_64-w64-mingw32 builds reproducible (Carl Dong) c4cce00eac691625b78b92f7dba0b7f57def19e5 guix: Remove dead links from README. (Carl Dong) df953a4c9a6143f45864757b706c88b6fa70545a guix: Appease shellcheck. (Carl Dong) 91897c95e191d293eb27d8af15cbeafc5b8f3895 guix: Improve guix-build.sh documentation (Carl Dong) 570d769c6c59b9f6d1a2b95b2ed60432cb33b3ba guix: Build support for Windows (Carl Dong) Pull request description: ~~Based on: https://github.com/bitcoin/bitcoin/pull/16519~~ Based on: #17933 (Time Machines are... shall we say... superior :grin:) This PR allows us to perform Guix builds for the `x86_64-w64-mingw32` target. We do this _without_ splitting up the build script like we do in Gitian by using this newfangled alien technology called `case` statements. (This is WIP and might be changed to `if` statements soon) ACKs for top commit: fanquake: ACK a35e3235891d35daa167116cc70340140e883f06 2/3 Tree-SHA512: c471951c23eb2cda919a71285d8b8f2580cb20f09d5db17b53e13dbd8813e01b3e7a83ea848e4913fd0f2bc12c6c133c5f76b54e65c0d89fed4dfd2e0be19875
2020-04-12 12:08:46 +02:00
(guix build-system gnu)
Merge bitcoin/bitcoin#21239: guix: Add codesignature attachment support for osx+win ee883201cf134952284632e9e9ae72bf1c8c792f guix: repro: Sort find output in libtool for gcc-8 (Carl Dong) ee0a67c32a8861eab650bf8894af06807578eba0 codesigning: Use SHA256 as digest for osslsigncode (Windows) (Carl Dong) 38eb91eb0616ed6dbe34c23e11588d130fef07f8 guix: Add codesigning functionality (Carl Dong) bac2690e6f683fcedb883fe1d32f3c33c628a141 guix: Package codesigning tools (Carl Dong) 0a2176d47767972e4cd5ed302c1dbeedece1708b guix: Reindent existing manifest.scm (Carl Dong) c090a3e9238ba2df07875b4708e908d8dca4ed9b Makefile.am: use APP_DIST_DIR instead of hard-coding dist (Carl Dong) Pull request description: This is the last PR before we reach feature-parity with the Gitian process! Note: I tried using the `Makefile` inside the distsrc to make the dmg instead of manually listing out the commands, but `make` seems to want to re-make a lot of other files which broke the dmg. The workflow looks something like this: 1. `env [ FOO=bar... ] ./contrib/guix/guix-build` (add additional env vars as necessary) 2. Codesigners only: 1. Copy `guix-build-<short-id>/output/x86_64-apple-darwin18/bitcoin-<short-id>-osx-unsigned.tar.gz` and `guix-build-<short-id>/output/x86_64-w64-mingw32/bitcoin-<short-id>-win-unsigned.tar.gz` to signing computer 2. Codesign with `./detached-sig-create.sh` inside the tarball 3. Upload contents of `signature-{osx,win}.tar.gz` to https://github.com/bitcoin-core/bitcoin-detached-sigs (as a new tag) 3. Checkout new tag for `bitcoin-core/bitcoin-detached-sigs` with the detached signatures 4. `env [ FOO=bar... ] DETACHED_SIGS_REPO=<path/to/bitcoin-detached-sigs> ./contrib/guix/guix-codesign` (modify env vars as necessary) 5. Make sure `guix.sigs` is cloned and updated 6. `env GUIX_SIGS_REPO=<path/to/guix.sigs> SIGNER=0x96AB007F1A7ED999=dongcarl ./contrib/guix/guix-attest` (modify env vars as necessary) 7. Commit your new signatures and SHA256SUMS in `guix.sigs` 8. Optionally, after there are multiple signatures in `guix.sigs`: `env GUIX_SIGS_REPO=<path/to/guix.sigs> ./contrib/guix/guix-verify` ACKs for top commit: laanwj: Tested ACK ee883201cf134952284632e9e9ae72bf1c8c792f achow101: ACK ee883201cf134952284632e9e9ae72bf1c8c792f Tree-SHA512: e812a07a5f19f900600c70cb9c717769ef544a6c0c12760b5558b76b6b37df863257f3dbf38b0757e6e06e334470267e94c9f2bdbc27409d6837b1a0bfc6acbc
2021-05-24 15:31:05 +02:00
(guix build-system python)
Merge #15277: contrib: Enable building in Guix containers 751549b52a9a4cd27389d807ae67f02bbb39cd7f contrib: guix: Additional clarifications re: substitutes (Carl Dong) cd3e947f50db7cfe05c05b368c25742193729a62 contrib: guix: Various improvements. (Carl Dong) 8dff3e48a9e03299468ed3b342642f01f70da9db contrib: guix: Clarify SOURCE_DATE_EPOCH. (Carl Dong) 3e80ec3ea9691c7c89173de922a113e643fe976b contrib: Add deterministic Guix builds. (Carl Dong) Pull request description: ~~**This post is kept updated as this project progresses. Use this [latest update link](https://github.com/bitcoin/bitcoin/pull/15277#issuecomment-497303718) to see what's new.**~~ Please read the `README.md`. ----- ### Guix Introduction This PR enables building bitcoin in Guix containers. [Guix](https://www.gnu.org/software/guix/manual/en/html_node/Features.html) is a transactional package manager much like Nix, but unlike Nix, it has more of a focus on [bootstrappability](https://www.gnu.org/software/guix/manual/en/html_node/Bootstrapping.html) and [reproducibility](https://www.gnu.org/software/guix/blog/tags/reproducible-builds/) which are attractive for security-sensitive projects like bitcoin. ### Guix Build Walkthrough Please read the `README.md`. [Old instructions no. 4](https://github.com/bitcoin/bitcoin/pull/15277#issuecomment-497303718) [Old instructions no. 3](https://github.com/bitcoin/bitcoin/pull/15277#issuecomment-493827011) [Old instructions no. 2](https://github.com/bitcoin/bitcoin/pull/15277#issuecomment-471658439) <details> <summary>Old instructions no. 1</summary> In this PR, we define a Guix [manifest](https://www.gnu.org/software/guix/manual/en/html_node/Invoking-guix-package.html#profile_002dmanifest) in `contrib/guix/manifest.scm`, which declares what packages we want in our environment. We can then invoke ``` guix environment --manifest=contrib/guix/manifest.scm --container --pure --no-grafts --no-substitutes ``` To have Guix: 1. Build an environment containing the packages we defined in our `contrib/guix/manifest.scm` manifest from the Guix bootstrap binaries (see [bootstrappability](https://www.gnu.org/software/guix/manual/en/html_node/Bootstrapping.html) for more details). 2. Start a container with that environment that has no network access, and no access to the host's filesystem except to the `pwd` that it was started in. 3. Drop you into a shell in that container. > Note: if you don't want to wait hours for Guix to build the entire world from scratch, you can eliminate the `--no-substitutes` option to have Guix download from available binary sources. Note that this convenience doesn't necessarily compromise your security, as you can check that a package was built correctly after the fact using `guix build --check <packagename>` Therefore, we can perform a build of bitcoin much like in Gitian by invoking the following: ``` make -C depends -j"$(nproc)" download && \ cat contrib/guix/build.sh | guix environment --manifest=contrib/guix/manifest.scm --container --pure --no-grafts --no-substitutes ``` We don't include `make -C depends -j"$(nproc)" download` inside `contrib/guix/build.sh` because `contrib/guix/build.sh` is run inside the container, which has no network access (which is a good thing). </details> ### Rationale I believe that this represents a substantial improvement for the "supply chain security" of bitcoin because: 1. We no longer have to rely on Ubuntu for our build environment for our releases ([oh the horror](https://github.com/bitcoin/bitcoin/blob/72bd4ab867e3be0d8410403d9641c08288d343e3/contrib/gitian-descriptors/gitian-linux.yml#L10)), because Guix builds everything about the container, we can perform this on almost any Linux distro/system. 2. It is now much easier to determine what trusted binaries are in our supply chain, and even make a nice visualization! (see [bootstrappability](https://www.gnu.org/software/guix/manual/en/html_node/Bootstrapping.html)). 3. There is active effort among Guix folks to minimize the number of trusted binaries even further. OriansJ's [stage0](https://github.com/oriansj/stage0), and janneke's [Mes](https://www.gnu.org/software/mes/) all aim to achieve [reduced binary boostrap](http://joyofsource.com/reduced-binary-seed-bootstrap.html) for Guix. In fact, I believe if OriansJ gets his way, we will end up some day with only a single trusted binary: hex0 (a ~500 byte self-hosting hex assembler). ### Steps to Completion - [x] Successfully build bitcoin inside the Guix environment - [x] Make `check-symbols` pass - [x] Do the above but without nasty hacks - [x] Solve some of the more innocuous hacks - [ ] Make it cross-compile (HELP WANTED HERE) - [x] Linux - [x] x86_64-linux-gnu - [x] i686-linux-gnu - [x] aarch64-linux-gnu - [x] arm-linux-gnueabihf - [x] riscv64-linux-gnu - [ ] OS X - [ ] x86_64-apple-darwin14 - [ ] Windows - [ ] x86_64-w64-mingw32 - [ ] Maybe make importer for depends syntax - [ ] Document build process for future releases - [ ] Extra: Pin the revision of Guix that we build with with Guix [inferiors](https://www.gnu.org/software/guix/manual/en/html_node/Inferiors.html) ### Help Wanted [Old content no. 3](https://github.com/bitcoin/bitcoin/pull/15277#issuecomment-483318210) [Old content no. 2](https://github.com/bitcoin/bitcoin/pull/15277#issuecomment-471658439) <details> <summary>Old content no. 1</summary> As of now, the command described above to perform a build of bitcoin a lot like Gitian works, but fails at the `check-symbols` stage. This is because a few dynamic libraries are linked in that shouldn't be. Here's what `ldd src/bitcoind` looks like when built in a Guix container: ``` linux-vdso.so.1 (0x00007ffcc2d90000) libdl.so.2 => /gnu/store/h90vnqw0nwd0hhm1l5dgxsdrigddfmq4-glibc-2.28/lib/libdl.so.2 (0x00007fb7eda09000) librt.so.1 => /gnu/store/h90vnqw0nwd0hhm1l5dgxsdrigddfmq4-glibc-2.28/lib/librt.so.1 (0x00007fb7ed9ff000) libstdc++.so.6 => /gnu/store/4sqps8dczv3g7rwbdibfz6rf5jlk7w90-gcc-5.5.0-lib/lib/libstdc++.so.6 (0x00007fb7ed87c000) libpthread.so.0 => /gnu/store/h90vnqw0nwd0hhm1l5dgxsdrigddfmq4-glibc-2.28/lib/libpthread.so.0 (0x00007fb7ed85b000) libm.so.6 => /gnu/store/h90vnqw0nwd0hhm1l5dgxsdrigddfmq4-glibc-2.28/lib/libm.so.6 (0x00007fb7ed6da000) libgcc_s.so.1 => /gnu/store/4sqps8dczv3g7rwbdibfz6rf5jlk7w90-gcc-5.5.0-lib/lib/libgcc_s.so.1 (0x00007fb7ed6bf000) libc.so.6 => /gnu/store/h90vnqw0nwd0hhm1l5dgxsdrigddfmq4-glibc-2.28/lib/libc.so.6 (0x00007fb7ed506000) /gnu/store/h90vnqw0nwd0hhm1l5dgxsdrigddfmq4-glibc-2.28/lib/ld-linux-x86-64.so.2 => /usr/lib64/ld-linux-x86-64.so.2 (0x00007fb7ee3a0000) ``` And here's what it looks in one of our releases: ``` linux-vdso.so.1 (0x00007ffff52cd000) libpthread.so.0 => /usr/lib/libpthread.so.0 (0x00007f87726b4000) librt.so.1 => /usr/lib/librt.so.1 (0x00007f87726aa000) libm.so.6 => /usr/lib/libm.so.6 (0x00007f8772525000) libgcc_s.so.1 => /usr/lib/libgcc_s.so.1 (0x00007f877250b000) libc.so.6 => /usr/lib/libc.so.6 (0x00007f8772347000) /lib64/ld-linux-x86-64.so.2 => /usr/lib64/ld-linux-x86-64.so.2 (0x00007f8773392000) ``` ~~I suspect it is because my script does not apply the gitian-input patches [described in the release process](https://github.com/bitcoin/bitcoin/blob/master/doc/release-process.md#fetch-and-create-inputs-first-time-or-when-dependency-versions-change) but there is no description as to how these patches are applied.~~ It might also be something else entirely. Edit: It is something else. It appears that the gitian inputs are only used by [`gitian-win-signer.yml`](https://github.com/bitcoin/bitcoin/blob/d6e700e40f861ddd6743f4d13f0d6f6bc19093c2/contrib/gitian-descriptors/gitian-win-signer.yml#L14) </details> ### How to Help 1. Install Guix on your distro either [from source](https://www.gnu.org/software/guix/manual/en/html_node/Requirements.html) or perform a [binary installation](https://www.gnu.org/software/guix/manual/en/html_node/Binary-Installation.html#Binary-Installation) 2. Try out my branch and the command described above! ACKs for top commit: MarcoFalke: Thanks for the replies. ACK 751549b52a9a4cd27389d807ae67f02bbb39cd7f laanwj: ACK 751549b52a9a4cd27389d807ae67f02bbb39cd7f Tree-SHA512: 50e6ab58c6bda9a67125b6271daf7eff0ca57d0efa8941ed3cd951e5bf78b31552fc5e537b1e1bcf2d3cc918c63adf19d685aa117a0f851024dc67e697890a8d
2019-07-12 19:23:30 +02:00
(guix build-system trivial)
(guix download)
Merge #15277: contrib: Enable building in Guix containers 751549b52a9a4cd27389d807ae67f02bbb39cd7f contrib: guix: Additional clarifications re: substitutes (Carl Dong) cd3e947f50db7cfe05c05b368c25742193729a62 contrib: guix: Various improvements. (Carl Dong) 8dff3e48a9e03299468ed3b342642f01f70da9db contrib: guix: Clarify SOURCE_DATE_EPOCH. (Carl Dong) 3e80ec3ea9691c7c89173de922a113e643fe976b contrib: Add deterministic Guix builds. (Carl Dong) Pull request description: ~~**This post is kept updated as this project progresses. Use this [latest update link](https://github.com/bitcoin/bitcoin/pull/15277#issuecomment-497303718) to see what's new.**~~ Please read the `README.md`. ----- ### Guix Introduction This PR enables building bitcoin in Guix containers. [Guix](https://www.gnu.org/software/guix/manual/en/html_node/Features.html) is a transactional package manager much like Nix, but unlike Nix, it has more of a focus on [bootstrappability](https://www.gnu.org/software/guix/manual/en/html_node/Bootstrapping.html) and [reproducibility](https://www.gnu.org/software/guix/blog/tags/reproducible-builds/) which are attractive for security-sensitive projects like bitcoin. ### Guix Build Walkthrough Please read the `README.md`. [Old instructions no. 4](https://github.com/bitcoin/bitcoin/pull/15277#issuecomment-497303718) [Old instructions no. 3](https://github.com/bitcoin/bitcoin/pull/15277#issuecomment-493827011) [Old instructions no. 2](https://github.com/bitcoin/bitcoin/pull/15277#issuecomment-471658439) <details> <summary>Old instructions no. 1</summary> In this PR, we define a Guix [manifest](https://www.gnu.org/software/guix/manual/en/html_node/Invoking-guix-package.html#profile_002dmanifest) in `contrib/guix/manifest.scm`, which declares what packages we want in our environment. We can then invoke ``` guix environment --manifest=contrib/guix/manifest.scm --container --pure --no-grafts --no-substitutes ``` To have Guix: 1. Build an environment containing the packages we defined in our `contrib/guix/manifest.scm` manifest from the Guix bootstrap binaries (see [bootstrappability](https://www.gnu.org/software/guix/manual/en/html_node/Bootstrapping.html) for more details). 2. Start a container with that environment that has no network access, and no access to the host's filesystem except to the `pwd` that it was started in. 3. Drop you into a shell in that container. > Note: if you don't want to wait hours for Guix to build the entire world from scratch, you can eliminate the `--no-substitutes` option to have Guix download from available binary sources. Note that this convenience doesn't necessarily compromise your security, as you can check that a package was built correctly after the fact using `guix build --check <packagename>` Therefore, we can perform a build of bitcoin much like in Gitian by invoking the following: ``` make -C depends -j"$(nproc)" download && \ cat contrib/guix/build.sh | guix environment --manifest=contrib/guix/manifest.scm --container --pure --no-grafts --no-substitutes ``` We don't include `make -C depends -j"$(nproc)" download` inside `contrib/guix/build.sh` because `contrib/guix/build.sh` is run inside the container, which has no network access (which is a good thing). </details> ### Rationale I believe that this represents a substantial improvement for the "supply chain security" of bitcoin because: 1. We no longer have to rely on Ubuntu for our build environment for our releases ([oh the horror](https://github.com/bitcoin/bitcoin/blob/72bd4ab867e3be0d8410403d9641c08288d343e3/contrib/gitian-descriptors/gitian-linux.yml#L10)), because Guix builds everything about the container, we can perform this on almost any Linux distro/system. 2. It is now much easier to determine what trusted binaries are in our supply chain, and even make a nice visualization! (see [bootstrappability](https://www.gnu.org/software/guix/manual/en/html_node/Bootstrapping.html)). 3. There is active effort among Guix folks to minimize the number of trusted binaries even further. OriansJ's [stage0](https://github.com/oriansj/stage0), and janneke's [Mes](https://www.gnu.org/software/mes/) all aim to achieve [reduced binary boostrap](http://joyofsource.com/reduced-binary-seed-bootstrap.html) for Guix. In fact, I believe if OriansJ gets his way, we will end up some day with only a single trusted binary: hex0 (a ~500 byte self-hosting hex assembler). ### Steps to Completion - [x] Successfully build bitcoin inside the Guix environment - [x] Make `check-symbols` pass - [x] Do the above but without nasty hacks - [x] Solve some of the more innocuous hacks - [ ] Make it cross-compile (HELP WANTED HERE) - [x] Linux - [x] x86_64-linux-gnu - [x] i686-linux-gnu - [x] aarch64-linux-gnu - [x] arm-linux-gnueabihf - [x] riscv64-linux-gnu - [ ] OS X - [ ] x86_64-apple-darwin14 - [ ] Windows - [ ] x86_64-w64-mingw32 - [ ] Maybe make importer for depends syntax - [ ] Document build process for future releases - [ ] Extra: Pin the revision of Guix that we build with with Guix [inferiors](https://www.gnu.org/software/guix/manual/en/html_node/Inferiors.html) ### Help Wanted [Old content no. 3](https://github.com/bitcoin/bitcoin/pull/15277#issuecomment-483318210) [Old content no. 2](https://github.com/bitcoin/bitcoin/pull/15277#issuecomment-471658439) <details> <summary>Old content no. 1</summary> As of now, the command described above to perform a build of bitcoin a lot like Gitian works, but fails at the `check-symbols` stage. This is because a few dynamic libraries are linked in that shouldn't be. Here's what `ldd src/bitcoind` looks like when built in a Guix container: ``` linux-vdso.so.1 (0x00007ffcc2d90000) libdl.so.2 => /gnu/store/h90vnqw0nwd0hhm1l5dgxsdrigddfmq4-glibc-2.28/lib/libdl.so.2 (0x00007fb7eda09000) librt.so.1 => /gnu/store/h90vnqw0nwd0hhm1l5dgxsdrigddfmq4-glibc-2.28/lib/librt.so.1 (0x00007fb7ed9ff000) libstdc++.so.6 => /gnu/store/4sqps8dczv3g7rwbdibfz6rf5jlk7w90-gcc-5.5.0-lib/lib/libstdc++.so.6 (0x00007fb7ed87c000) libpthread.so.0 => /gnu/store/h90vnqw0nwd0hhm1l5dgxsdrigddfmq4-glibc-2.28/lib/libpthread.so.0 (0x00007fb7ed85b000) libm.so.6 => /gnu/store/h90vnqw0nwd0hhm1l5dgxsdrigddfmq4-glibc-2.28/lib/libm.so.6 (0x00007fb7ed6da000) libgcc_s.so.1 => /gnu/store/4sqps8dczv3g7rwbdibfz6rf5jlk7w90-gcc-5.5.0-lib/lib/libgcc_s.so.1 (0x00007fb7ed6bf000) libc.so.6 => /gnu/store/h90vnqw0nwd0hhm1l5dgxsdrigddfmq4-glibc-2.28/lib/libc.so.6 (0x00007fb7ed506000) /gnu/store/h90vnqw0nwd0hhm1l5dgxsdrigddfmq4-glibc-2.28/lib/ld-linux-x86-64.so.2 => /usr/lib64/ld-linux-x86-64.so.2 (0x00007fb7ee3a0000) ``` And here's what it looks in one of our releases: ``` linux-vdso.so.1 (0x00007ffff52cd000) libpthread.so.0 => /usr/lib/libpthread.so.0 (0x00007f87726b4000) librt.so.1 => /usr/lib/librt.so.1 (0x00007f87726aa000) libm.so.6 => /usr/lib/libm.so.6 (0x00007f8772525000) libgcc_s.so.1 => /usr/lib/libgcc_s.so.1 (0x00007f877250b000) libc.so.6 => /usr/lib/libc.so.6 (0x00007f8772347000) /lib64/ld-linux-x86-64.so.2 => /usr/lib64/ld-linux-x86-64.so.2 (0x00007f8773392000) ``` ~~I suspect it is because my script does not apply the gitian-input patches [described in the release process](https://github.com/bitcoin/bitcoin/blob/master/doc/release-process.md#fetch-and-create-inputs-first-time-or-when-dependency-versions-change) but there is no description as to how these patches are applied.~~ It might also be something else entirely. Edit: It is something else. It appears that the gitian inputs are only used by [`gitian-win-signer.yml`](https://github.com/bitcoin/bitcoin/blob/d6e700e40f861ddd6743f4d13f0d6f6bc19093c2/contrib/gitian-descriptors/gitian-win-signer.yml#L14) </details> ### How to Help 1. Install Guix on your distro either [from source](https://www.gnu.org/software/guix/manual/en/html_node/Requirements.html) or perform a [binary installation](https://www.gnu.org/software/guix/manual/en/html_node/Binary-Installation.html#Binary-Installation) 2. Try out my branch and the command described above! ACKs for top commit: MarcoFalke: Thanks for the replies. ACK 751549b52a9a4cd27389d807ae67f02bbb39cd7f laanwj: ACK 751549b52a9a4cd27389d807ae67f02bbb39cd7f Tree-SHA512: 50e6ab58c6bda9a67125b6271daf7eff0ca57d0efa8941ed3cd951e5bf78b31552fc5e537b1e1bcf2d3cc918c63adf19d685aa117a0f851024dc67e697890a8d
2019-07-12 19:23:30 +02:00
(guix gexp)
Merge bitcoin/bitcoin#21239: guix: Add codesignature attachment support for osx+win ee883201cf134952284632e9e9ae72bf1c8c792f guix: repro: Sort find output in libtool for gcc-8 (Carl Dong) ee0a67c32a8861eab650bf8894af06807578eba0 codesigning: Use SHA256 as digest for osslsigncode (Windows) (Carl Dong) 38eb91eb0616ed6dbe34c23e11588d130fef07f8 guix: Add codesigning functionality (Carl Dong) bac2690e6f683fcedb883fe1d32f3c33c628a141 guix: Package codesigning tools (Carl Dong) 0a2176d47767972e4cd5ed302c1dbeedece1708b guix: Reindent existing manifest.scm (Carl Dong) c090a3e9238ba2df07875b4708e908d8dca4ed9b Makefile.am: use APP_DIST_DIR instead of hard-coding dist (Carl Dong) Pull request description: This is the last PR before we reach feature-parity with the Gitian process! Note: I tried using the `Makefile` inside the distsrc to make the dmg instead of manually listing out the commands, but `make` seems to want to re-make a lot of other files which broke the dmg. The workflow looks something like this: 1. `env [ FOO=bar... ] ./contrib/guix/guix-build` (add additional env vars as necessary) 2. Codesigners only: 1. Copy `guix-build-<short-id>/output/x86_64-apple-darwin18/bitcoin-<short-id>-osx-unsigned.tar.gz` and `guix-build-<short-id>/output/x86_64-w64-mingw32/bitcoin-<short-id>-win-unsigned.tar.gz` to signing computer 2. Codesign with `./detached-sig-create.sh` inside the tarball 3. Upload contents of `signature-{osx,win}.tar.gz` to https://github.com/bitcoin-core/bitcoin-detached-sigs (as a new tag) 3. Checkout new tag for `bitcoin-core/bitcoin-detached-sigs` with the detached signatures 4. `env [ FOO=bar... ] DETACHED_SIGS_REPO=<path/to/bitcoin-detached-sigs> ./contrib/guix/guix-codesign` (modify env vars as necessary) 5. Make sure `guix.sigs` is cloned and updated 6. `env GUIX_SIGS_REPO=<path/to/guix.sigs> SIGNER=0x96AB007F1A7ED999=dongcarl ./contrib/guix/guix-attest` (modify env vars as necessary) 7. Commit your new signatures and SHA256SUMS in `guix.sigs` 8. Optionally, after there are multiple signatures in `guix.sigs`: `env GUIX_SIGS_REPO=<path/to/guix.sigs> ./contrib/guix/guix-verify` ACKs for top commit: laanwj: Tested ACK ee883201cf134952284632e9e9ae72bf1c8c792f achow101: ACK ee883201cf134952284632e9e9ae72bf1c8c792f Tree-SHA512: e812a07a5f19f900600c70cb9c717769ef544a6c0c12760b5558b76b6b37df863257f3dbf38b0757e6e06e334470267e94c9f2bdbc27409d6837b1a0bfc6acbc
2021-05-24 15:31:05 +02:00
(guix git-download)
((guix licenses) #:prefix license:)
Merge #15277: contrib: Enable building in Guix containers 751549b52a9a4cd27389d807ae67f02bbb39cd7f contrib: guix: Additional clarifications re: substitutes (Carl Dong) cd3e947f50db7cfe05c05b368c25742193729a62 contrib: guix: Various improvements. (Carl Dong) 8dff3e48a9e03299468ed3b342642f01f70da9db contrib: guix: Clarify SOURCE_DATE_EPOCH. (Carl Dong) 3e80ec3ea9691c7c89173de922a113e643fe976b contrib: Add deterministic Guix builds. (Carl Dong) Pull request description: ~~**This post is kept updated as this project progresses. Use this [latest update link](https://github.com/bitcoin/bitcoin/pull/15277#issuecomment-497303718) to see what's new.**~~ Please read the `README.md`. ----- ### Guix Introduction This PR enables building bitcoin in Guix containers. [Guix](https://www.gnu.org/software/guix/manual/en/html_node/Features.html) is a transactional package manager much like Nix, but unlike Nix, it has more of a focus on [bootstrappability](https://www.gnu.org/software/guix/manual/en/html_node/Bootstrapping.html) and [reproducibility](https://www.gnu.org/software/guix/blog/tags/reproducible-builds/) which are attractive for security-sensitive projects like bitcoin. ### Guix Build Walkthrough Please read the `README.md`. [Old instructions no. 4](https://github.com/bitcoin/bitcoin/pull/15277#issuecomment-497303718) [Old instructions no. 3](https://github.com/bitcoin/bitcoin/pull/15277#issuecomment-493827011) [Old instructions no. 2](https://github.com/bitcoin/bitcoin/pull/15277#issuecomment-471658439) <details> <summary>Old instructions no. 1</summary> In this PR, we define a Guix [manifest](https://www.gnu.org/software/guix/manual/en/html_node/Invoking-guix-package.html#profile_002dmanifest) in `contrib/guix/manifest.scm`, which declares what packages we want in our environment. We can then invoke ``` guix environment --manifest=contrib/guix/manifest.scm --container --pure --no-grafts --no-substitutes ``` To have Guix: 1. Build an environment containing the packages we defined in our `contrib/guix/manifest.scm` manifest from the Guix bootstrap binaries (see [bootstrappability](https://www.gnu.org/software/guix/manual/en/html_node/Bootstrapping.html) for more details). 2. Start a container with that environment that has no network access, and no access to the host's filesystem except to the `pwd` that it was started in. 3. Drop you into a shell in that container. > Note: if you don't want to wait hours for Guix to build the entire world from scratch, you can eliminate the `--no-substitutes` option to have Guix download from available binary sources. Note that this convenience doesn't necessarily compromise your security, as you can check that a package was built correctly after the fact using `guix build --check <packagename>` Therefore, we can perform a build of bitcoin much like in Gitian by invoking the following: ``` make -C depends -j"$(nproc)" download && \ cat contrib/guix/build.sh | guix environment --manifest=contrib/guix/manifest.scm --container --pure --no-grafts --no-substitutes ``` We don't include `make -C depends -j"$(nproc)" download` inside `contrib/guix/build.sh` because `contrib/guix/build.sh` is run inside the container, which has no network access (which is a good thing). </details> ### Rationale I believe that this represents a substantial improvement for the "supply chain security" of bitcoin because: 1. We no longer have to rely on Ubuntu for our build environment for our releases ([oh the horror](https://github.com/bitcoin/bitcoin/blob/72bd4ab867e3be0d8410403d9641c08288d343e3/contrib/gitian-descriptors/gitian-linux.yml#L10)), because Guix builds everything about the container, we can perform this on almost any Linux distro/system. 2. It is now much easier to determine what trusted binaries are in our supply chain, and even make a nice visualization! (see [bootstrappability](https://www.gnu.org/software/guix/manual/en/html_node/Bootstrapping.html)). 3. There is active effort among Guix folks to minimize the number of trusted binaries even further. OriansJ's [stage0](https://github.com/oriansj/stage0), and janneke's [Mes](https://www.gnu.org/software/mes/) all aim to achieve [reduced binary boostrap](http://joyofsource.com/reduced-binary-seed-bootstrap.html) for Guix. In fact, I believe if OriansJ gets his way, we will end up some day with only a single trusted binary: hex0 (a ~500 byte self-hosting hex assembler). ### Steps to Completion - [x] Successfully build bitcoin inside the Guix environment - [x] Make `check-symbols` pass - [x] Do the above but without nasty hacks - [x] Solve some of the more innocuous hacks - [ ] Make it cross-compile (HELP WANTED HERE) - [x] Linux - [x] x86_64-linux-gnu - [x] i686-linux-gnu - [x] aarch64-linux-gnu - [x] arm-linux-gnueabihf - [x] riscv64-linux-gnu - [ ] OS X - [ ] x86_64-apple-darwin14 - [ ] Windows - [ ] x86_64-w64-mingw32 - [ ] Maybe make importer for depends syntax - [ ] Document build process for future releases - [ ] Extra: Pin the revision of Guix that we build with with Guix [inferiors](https://www.gnu.org/software/guix/manual/en/html_node/Inferiors.html) ### Help Wanted [Old content no. 3](https://github.com/bitcoin/bitcoin/pull/15277#issuecomment-483318210) [Old content no. 2](https://github.com/bitcoin/bitcoin/pull/15277#issuecomment-471658439) <details> <summary>Old content no. 1</summary> As of now, the command described above to perform a build of bitcoin a lot like Gitian works, but fails at the `check-symbols` stage. This is because a few dynamic libraries are linked in that shouldn't be. Here's what `ldd src/bitcoind` looks like when built in a Guix container: ``` linux-vdso.so.1 (0x00007ffcc2d90000) libdl.so.2 => /gnu/store/h90vnqw0nwd0hhm1l5dgxsdrigddfmq4-glibc-2.28/lib/libdl.so.2 (0x00007fb7eda09000) librt.so.1 => /gnu/store/h90vnqw0nwd0hhm1l5dgxsdrigddfmq4-glibc-2.28/lib/librt.so.1 (0x00007fb7ed9ff000) libstdc++.so.6 => /gnu/store/4sqps8dczv3g7rwbdibfz6rf5jlk7w90-gcc-5.5.0-lib/lib/libstdc++.so.6 (0x00007fb7ed87c000) libpthread.so.0 => /gnu/store/h90vnqw0nwd0hhm1l5dgxsdrigddfmq4-glibc-2.28/lib/libpthread.so.0 (0x00007fb7ed85b000) libm.so.6 => /gnu/store/h90vnqw0nwd0hhm1l5dgxsdrigddfmq4-glibc-2.28/lib/libm.so.6 (0x00007fb7ed6da000) libgcc_s.so.1 => /gnu/store/4sqps8dczv3g7rwbdibfz6rf5jlk7w90-gcc-5.5.0-lib/lib/libgcc_s.so.1 (0x00007fb7ed6bf000) libc.so.6 => /gnu/store/h90vnqw0nwd0hhm1l5dgxsdrigddfmq4-glibc-2.28/lib/libc.so.6 (0x00007fb7ed506000) /gnu/store/h90vnqw0nwd0hhm1l5dgxsdrigddfmq4-glibc-2.28/lib/ld-linux-x86-64.so.2 => /usr/lib64/ld-linux-x86-64.so.2 (0x00007fb7ee3a0000) ``` And here's what it looks in one of our releases: ``` linux-vdso.so.1 (0x00007ffff52cd000) libpthread.so.0 => /usr/lib/libpthread.so.0 (0x00007f87726b4000) librt.so.1 => /usr/lib/librt.so.1 (0x00007f87726aa000) libm.so.6 => /usr/lib/libm.so.6 (0x00007f8772525000) libgcc_s.so.1 => /usr/lib/libgcc_s.so.1 (0x00007f877250b000) libc.so.6 => /usr/lib/libc.so.6 (0x00007f8772347000) /lib64/ld-linux-x86-64.so.2 => /usr/lib64/ld-linux-x86-64.so.2 (0x00007f8773392000) ``` ~~I suspect it is because my script does not apply the gitian-input patches [described in the release process](https://github.com/bitcoin/bitcoin/blob/master/doc/release-process.md#fetch-and-create-inputs-first-time-or-when-dependency-versions-change) but there is no description as to how these patches are applied.~~ It might also be something else entirely. Edit: It is something else. It appears that the gitian inputs are only used by [`gitian-win-signer.yml`](https://github.com/bitcoin/bitcoin/blob/d6e700e40f861ddd6743f4d13f0d6f6bc19093c2/contrib/gitian-descriptors/gitian-win-signer.yml#L14) </details> ### How to Help 1. Install Guix on your distro either [from source](https://www.gnu.org/software/guix/manual/en/html_node/Requirements.html) or perform a [binary installation](https://www.gnu.org/software/guix/manual/en/html_node/Binary-Installation.html#Binary-Installation) 2. Try out my branch and the command described above! ACKs for top commit: MarcoFalke: Thanks for the replies. ACK 751549b52a9a4cd27389d807ae67f02bbb39cd7f laanwj: ACK 751549b52a9a4cd27389d807ae67f02bbb39cd7f Tree-SHA512: 50e6ab58c6bda9a67125b6271daf7eff0ca57d0efa8941ed3cd951e5bf78b31552fc5e537b1e1bcf2d3cc918c63adf19d685aa117a0f851024dc67e697890a8d
2019-07-12 19:23:30 +02:00
(guix packages)
(guix profiles)
(guix utils))
(define-syntax-rule (search-our-patches file-name ...)
"Return the list of absolute file names corresponding to each
FILE-NAME found in ./patches relative to the current file."
(parameterize
((%patch-path (list (string-append (dirname (current-filename)) "/patches"))))
(list (search-patch file-name) ...)))
Merge #17595: guix: Enable building for `x86_64-w64-mingw32` target a35e3235891d35daa167116cc70340140e883f06 guix: Appease travis. (Carl Dong) 0b66d22da5f53640e22f05adf880782c613e6d0f guix: Use gcc-9 for mingw-w64 instead of 8 (Carl Dong) ba0b99bdd613ba7f17c6247ece3001e1b44759a3 guix: Don't set MINGW_HAS_SECURE_API CFLAG in depends (Carl Dong) 93439a71eda49fb69f1e82966a23a946733aa6fa guix: Bump to upstream commit with mingw-w64 changes (Carl Dong) 35a96792dda9e78165b1598aeac7b2ab759e7be5 guix: Check mingw symbols, improve SSP fix docs (Carl Dong) 449d8fe25bbe25daacfc67aa89ca32b0a3254c5a guix: Expand on INT trap message (Carl Dong) 3f1f03c67a8e9edf487f08d272adb18b0a3942c8 guix: Spelling fixes (Carl Dong) ff821dd2a1c600488d11e7d9a20e9179ecc9144b guix: Reinstate make-ssp-fixed-gcc (Carl Dong) 360a9e0ad50a36ec79a1a160dbed3966689fd41c guix: Bump time-machine for mingw-w64 patches (Carl Dong) 93e41b7e3b54c17fd1b4c61ee95fc0dc2827e954 guix: Use gcc-8 for mingw-w64 instead of 7 (Carl Dong) ef4f7e4c45c60a69406134122f091c77c6ef740f guix: Set the well-known timezone env var (Carl Dong) acf4b3b3b5accf60a19441a0298ef27001b78e72 guix: Make x86_64-w64-mingw32 builds reproducible (Carl Dong) c4cce00eac691625b78b92f7dba0b7f57def19e5 guix: Remove dead links from README. (Carl Dong) df953a4c9a6143f45864757b706c88b6fa70545a guix: Appease shellcheck. (Carl Dong) 91897c95e191d293eb27d8af15cbeafc5b8f3895 guix: Improve guix-build.sh documentation (Carl Dong) 570d769c6c59b9f6d1a2b95b2ed60432cb33b3ba guix: Build support for Windows (Carl Dong) Pull request description: ~~Based on: https://github.com/bitcoin/bitcoin/pull/16519~~ Based on: #17933 (Time Machines are... shall we say... superior :grin:) This PR allows us to perform Guix builds for the `x86_64-w64-mingw32` target. We do this _without_ splitting up the build script like we do in Gitian by using this newfangled alien technology called `case` statements. (This is WIP and might be changed to `if` statements soon) ACKs for top commit: fanquake: ACK a35e3235891d35daa167116cc70340140e883f06 2/3 Tree-SHA512: c471951c23eb2cda919a71285d8b8f2580cb20f09d5db17b53e13dbd8813e01b3e7a83ea848e4913fd0f2bc12c6c133c5f76b54e65c0d89fed4dfd2e0be19875
2020-04-12 12:08:46 +02:00
(define (make-ssp-fixed-gcc xgcc)
"Given a XGCC package, return a modified package that uses the SSP function
from glibc instead of from libssp.so. Our `symbol-check' script will complain if
we link against libssp.so, and thus will ensure that this works properly.
Taken from:
http://www.linuxfromscratch.org/hlfs/view/development/chapter05/gcc-pass1.html"
(package
Merge bitcoin/bitcoin#21239: guix: Add codesignature attachment support for osx+win ee883201cf134952284632e9e9ae72bf1c8c792f guix: repro: Sort find output in libtool for gcc-8 (Carl Dong) ee0a67c32a8861eab650bf8894af06807578eba0 codesigning: Use SHA256 as digest for osslsigncode (Windows) (Carl Dong) 38eb91eb0616ed6dbe34c23e11588d130fef07f8 guix: Add codesigning functionality (Carl Dong) bac2690e6f683fcedb883fe1d32f3c33c628a141 guix: Package codesigning tools (Carl Dong) 0a2176d47767972e4cd5ed302c1dbeedece1708b guix: Reindent existing manifest.scm (Carl Dong) c090a3e9238ba2df07875b4708e908d8dca4ed9b Makefile.am: use APP_DIST_DIR instead of hard-coding dist (Carl Dong) Pull request description: This is the last PR before we reach feature-parity with the Gitian process! Note: I tried using the `Makefile` inside the distsrc to make the dmg instead of manually listing out the commands, but `make` seems to want to re-make a lot of other files which broke the dmg. The workflow looks something like this: 1. `env [ FOO=bar... ] ./contrib/guix/guix-build` (add additional env vars as necessary) 2. Codesigners only: 1. Copy `guix-build-<short-id>/output/x86_64-apple-darwin18/bitcoin-<short-id>-osx-unsigned.tar.gz` and `guix-build-<short-id>/output/x86_64-w64-mingw32/bitcoin-<short-id>-win-unsigned.tar.gz` to signing computer 2. Codesign with `./detached-sig-create.sh` inside the tarball 3. Upload contents of `signature-{osx,win}.tar.gz` to https://github.com/bitcoin-core/bitcoin-detached-sigs (as a new tag) 3. Checkout new tag for `bitcoin-core/bitcoin-detached-sigs` with the detached signatures 4. `env [ FOO=bar... ] DETACHED_SIGS_REPO=<path/to/bitcoin-detached-sigs> ./contrib/guix/guix-codesign` (modify env vars as necessary) 5. Make sure `guix.sigs` is cloned and updated 6. `env GUIX_SIGS_REPO=<path/to/guix.sigs> SIGNER=0x96AB007F1A7ED999=dongcarl ./contrib/guix/guix-attest` (modify env vars as necessary) 7. Commit your new signatures and SHA256SUMS in `guix.sigs` 8. Optionally, after there are multiple signatures in `guix.sigs`: `env GUIX_SIGS_REPO=<path/to/guix.sigs> ./contrib/guix/guix-verify` ACKs for top commit: laanwj: Tested ACK ee883201cf134952284632e9e9ae72bf1c8c792f achow101: ACK ee883201cf134952284632e9e9ae72bf1c8c792f Tree-SHA512: e812a07a5f19f900600c70cb9c717769ef544a6c0c12760b5558b76b6b37df863257f3dbf38b0757e6e06e334470267e94c9f2bdbc27409d6837b1a0bfc6acbc
2021-05-24 15:31:05 +02:00
(inherit xgcc)
(arguments
(substitute-keyword-arguments (package-arguments xgcc)
((#:make-flags flags)
`(cons "gcc_cv_libc_provides_ssp=yes" ,flags))))))
Merge #17595: guix: Enable building for `x86_64-w64-mingw32` target a35e3235891d35daa167116cc70340140e883f06 guix: Appease travis. (Carl Dong) 0b66d22da5f53640e22f05adf880782c613e6d0f guix: Use gcc-9 for mingw-w64 instead of 8 (Carl Dong) ba0b99bdd613ba7f17c6247ece3001e1b44759a3 guix: Don't set MINGW_HAS_SECURE_API CFLAG in depends (Carl Dong) 93439a71eda49fb69f1e82966a23a946733aa6fa guix: Bump to upstream commit with mingw-w64 changes (Carl Dong) 35a96792dda9e78165b1598aeac7b2ab759e7be5 guix: Check mingw symbols, improve SSP fix docs (Carl Dong) 449d8fe25bbe25daacfc67aa89ca32b0a3254c5a guix: Expand on INT trap message (Carl Dong) 3f1f03c67a8e9edf487f08d272adb18b0a3942c8 guix: Spelling fixes (Carl Dong) ff821dd2a1c600488d11e7d9a20e9179ecc9144b guix: Reinstate make-ssp-fixed-gcc (Carl Dong) 360a9e0ad50a36ec79a1a160dbed3966689fd41c guix: Bump time-machine for mingw-w64 patches (Carl Dong) 93e41b7e3b54c17fd1b4c61ee95fc0dc2827e954 guix: Use gcc-8 for mingw-w64 instead of 7 (Carl Dong) ef4f7e4c45c60a69406134122f091c77c6ef740f guix: Set the well-known timezone env var (Carl Dong) acf4b3b3b5accf60a19441a0298ef27001b78e72 guix: Make x86_64-w64-mingw32 builds reproducible (Carl Dong) c4cce00eac691625b78b92f7dba0b7f57def19e5 guix: Remove dead links from README. (Carl Dong) df953a4c9a6143f45864757b706c88b6fa70545a guix: Appease shellcheck. (Carl Dong) 91897c95e191d293eb27d8af15cbeafc5b8f3895 guix: Improve guix-build.sh documentation (Carl Dong) 570d769c6c59b9f6d1a2b95b2ed60432cb33b3ba guix: Build support for Windows (Carl Dong) Pull request description: ~~Based on: https://github.com/bitcoin/bitcoin/pull/16519~~ Based on: #17933 (Time Machines are... shall we say... superior :grin:) This PR allows us to perform Guix builds for the `x86_64-w64-mingw32` target. We do this _without_ splitting up the build script like we do in Gitian by using this newfangled alien technology called `case` statements. (This is WIP and might be changed to `if` statements soon) ACKs for top commit: fanquake: ACK a35e3235891d35daa167116cc70340140e883f06 2/3 Tree-SHA512: c471951c23eb2cda919a71285d8b8f2580cb20f09d5db17b53e13dbd8813e01b3e7a83ea848e4913fd0f2bc12c6c133c5f76b54e65c0d89fed4dfd2e0be19875
2020-04-12 12:08:46 +02:00
Merge #15277: contrib: Enable building in Guix containers 751549b52a9a4cd27389d807ae67f02bbb39cd7f contrib: guix: Additional clarifications re: substitutes (Carl Dong) cd3e947f50db7cfe05c05b368c25742193729a62 contrib: guix: Various improvements. (Carl Dong) 8dff3e48a9e03299468ed3b342642f01f70da9db contrib: guix: Clarify SOURCE_DATE_EPOCH. (Carl Dong) 3e80ec3ea9691c7c89173de922a113e643fe976b contrib: Add deterministic Guix builds. (Carl Dong) Pull request description: ~~**This post is kept updated as this project progresses. Use this [latest update link](https://github.com/bitcoin/bitcoin/pull/15277#issuecomment-497303718) to see what's new.**~~ Please read the `README.md`. ----- ### Guix Introduction This PR enables building bitcoin in Guix containers. [Guix](https://www.gnu.org/software/guix/manual/en/html_node/Features.html) is a transactional package manager much like Nix, but unlike Nix, it has more of a focus on [bootstrappability](https://www.gnu.org/software/guix/manual/en/html_node/Bootstrapping.html) and [reproducibility](https://www.gnu.org/software/guix/blog/tags/reproducible-builds/) which are attractive for security-sensitive projects like bitcoin. ### Guix Build Walkthrough Please read the `README.md`. [Old instructions no. 4](https://github.com/bitcoin/bitcoin/pull/15277#issuecomment-497303718) [Old instructions no. 3](https://github.com/bitcoin/bitcoin/pull/15277#issuecomment-493827011) [Old instructions no. 2](https://github.com/bitcoin/bitcoin/pull/15277#issuecomment-471658439) <details> <summary>Old instructions no. 1</summary> In this PR, we define a Guix [manifest](https://www.gnu.org/software/guix/manual/en/html_node/Invoking-guix-package.html#profile_002dmanifest) in `contrib/guix/manifest.scm`, which declares what packages we want in our environment. We can then invoke ``` guix environment --manifest=contrib/guix/manifest.scm --container --pure --no-grafts --no-substitutes ``` To have Guix: 1. Build an environment containing the packages we defined in our `contrib/guix/manifest.scm` manifest from the Guix bootstrap binaries (see [bootstrappability](https://www.gnu.org/software/guix/manual/en/html_node/Bootstrapping.html) for more details). 2. Start a container with that environment that has no network access, and no access to the host's filesystem except to the `pwd` that it was started in. 3. Drop you into a shell in that container. > Note: if you don't want to wait hours for Guix to build the entire world from scratch, you can eliminate the `--no-substitutes` option to have Guix download from available binary sources. Note that this convenience doesn't necessarily compromise your security, as you can check that a package was built correctly after the fact using `guix build --check <packagename>` Therefore, we can perform a build of bitcoin much like in Gitian by invoking the following: ``` make -C depends -j"$(nproc)" download && \ cat contrib/guix/build.sh | guix environment --manifest=contrib/guix/manifest.scm --container --pure --no-grafts --no-substitutes ``` We don't include `make -C depends -j"$(nproc)" download` inside `contrib/guix/build.sh` because `contrib/guix/build.sh` is run inside the container, which has no network access (which is a good thing). </details> ### Rationale I believe that this represents a substantial improvement for the "supply chain security" of bitcoin because: 1. We no longer have to rely on Ubuntu for our build environment for our releases ([oh the horror](https://github.com/bitcoin/bitcoin/blob/72bd4ab867e3be0d8410403d9641c08288d343e3/contrib/gitian-descriptors/gitian-linux.yml#L10)), because Guix builds everything about the container, we can perform this on almost any Linux distro/system. 2. It is now much easier to determine what trusted binaries are in our supply chain, and even make a nice visualization! (see [bootstrappability](https://www.gnu.org/software/guix/manual/en/html_node/Bootstrapping.html)). 3. There is active effort among Guix folks to minimize the number of trusted binaries even further. OriansJ's [stage0](https://github.com/oriansj/stage0), and janneke's [Mes](https://www.gnu.org/software/mes/) all aim to achieve [reduced binary boostrap](http://joyofsource.com/reduced-binary-seed-bootstrap.html) for Guix. In fact, I believe if OriansJ gets his way, we will end up some day with only a single trusted binary: hex0 (a ~500 byte self-hosting hex assembler). ### Steps to Completion - [x] Successfully build bitcoin inside the Guix environment - [x] Make `check-symbols` pass - [x] Do the above but without nasty hacks - [x] Solve some of the more innocuous hacks - [ ] Make it cross-compile (HELP WANTED HERE) - [x] Linux - [x] x86_64-linux-gnu - [x] i686-linux-gnu - [x] aarch64-linux-gnu - [x] arm-linux-gnueabihf - [x] riscv64-linux-gnu - [ ] OS X - [ ] x86_64-apple-darwin14 - [ ] Windows - [ ] x86_64-w64-mingw32 - [ ] Maybe make importer for depends syntax - [ ] Document build process for future releases - [ ] Extra: Pin the revision of Guix that we build with with Guix [inferiors](https://www.gnu.org/software/guix/manual/en/html_node/Inferiors.html) ### Help Wanted [Old content no. 3](https://github.com/bitcoin/bitcoin/pull/15277#issuecomment-483318210) [Old content no. 2](https://github.com/bitcoin/bitcoin/pull/15277#issuecomment-471658439) <details> <summary>Old content no. 1</summary> As of now, the command described above to perform a build of bitcoin a lot like Gitian works, but fails at the `check-symbols` stage. This is because a few dynamic libraries are linked in that shouldn't be. Here's what `ldd src/bitcoind` looks like when built in a Guix container: ``` linux-vdso.so.1 (0x00007ffcc2d90000) libdl.so.2 => /gnu/store/h90vnqw0nwd0hhm1l5dgxsdrigddfmq4-glibc-2.28/lib/libdl.so.2 (0x00007fb7eda09000) librt.so.1 => /gnu/store/h90vnqw0nwd0hhm1l5dgxsdrigddfmq4-glibc-2.28/lib/librt.so.1 (0x00007fb7ed9ff000) libstdc++.so.6 => /gnu/store/4sqps8dczv3g7rwbdibfz6rf5jlk7w90-gcc-5.5.0-lib/lib/libstdc++.so.6 (0x00007fb7ed87c000) libpthread.so.0 => /gnu/store/h90vnqw0nwd0hhm1l5dgxsdrigddfmq4-glibc-2.28/lib/libpthread.so.0 (0x00007fb7ed85b000) libm.so.6 => /gnu/store/h90vnqw0nwd0hhm1l5dgxsdrigddfmq4-glibc-2.28/lib/libm.so.6 (0x00007fb7ed6da000) libgcc_s.so.1 => /gnu/store/4sqps8dczv3g7rwbdibfz6rf5jlk7w90-gcc-5.5.0-lib/lib/libgcc_s.so.1 (0x00007fb7ed6bf000) libc.so.6 => /gnu/store/h90vnqw0nwd0hhm1l5dgxsdrigddfmq4-glibc-2.28/lib/libc.so.6 (0x00007fb7ed506000) /gnu/store/h90vnqw0nwd0hhm1l5dgxsdrigddfmq4-glibc-2.28/lib/ld-linux-x86-64.so.2 => /usr/lib64/ld-linux-x86-64.so.2 (0x00007fb7ee3a0000) ``` And here's what it looks in one of our releases: ``` linux-vdso.so.1 (0x00007ffff52cd000) libpthread.so.0 => /usr/lib/libpthread.so.0 (0x00007f87726b4000) librt.so.1 => /usr/lib/librt.so.1 (0x00007f87726aa000) libm.so.6 => /usr/lib/libm.so.6 (0x00007f8772525000) libgcc_s.so.1 => /usr/lib/libgcc_s.so.1 (0x00007f877250b000) libc.so.6 => /usr/lib/libc.so.6 (0x00007f8772347000) /lib64/ld-linux-x86-64.so.2 => /usr/lib64/ld-linux-x86-64.so.2 (0x00007f8773392000) ``` ~~I suspect it is because my script does not apply the gitian-input patches [described in the release process](https://github.com/bitcoin/bitcoin/blob/master/doc/release-process.md#fetch-and-create-inputs-first-time-or-when-dependency-versions-change) but there is no description as to how these patches are applied.~~ It might also be something else entirely. Edit: It is something else. It appears that the gitian inputs are only used by [`gitian-win-signer.yml`](https://github.com/bitcoin/bitcoin/blob/d6e700e40f861ddd6743f4d13f0d6f6bc19093c2/contrib/gitian-descriptors/gitian-win-signer.yml#L14) </details> ### How to Help 1. Install Guix on your distro either [from source](https://www.gnu.org/software/guix/manual/en/html_node/Requirements.html) or perform a [binary installation](https://www.gnu.org/software/guix/manual/en/html_node/Binary-Installation.html#Binary-Installation) 2. Try out my branch and the command described above! ACKs for top commit: MarcoFalke: Thanks for the replies. ACK 751549b52a9a4cd27389d807ae67f02bbb39cd7f laanwj: ACK 751549b52a9a4cd27389d807ae67f02bbb39cd7f Tree-SHA512: 50e6ab58c6bda9a67125b6271daf7eff0ca57d0efa8941ed3cd951e5bf78b31552fc5e537b1e1bcf2d3cc918c63adf19d685aa117a0f851024dc67e697890a8d
2019-07-12 19:23:30 +02:00
(define (make-gcc-rpath-link xgcc)
"Given a XGCC package, return a modified package that replace each instance of
-rpath in the default system spec that's inserted by Guix with -rpath-link"
(package
Merge bitcoin/bitcoin#21239: guix: Add codesignature attachment support for osx+win ee883201cf134952284632e9e9ae72bf1c8c792f guix: repro: Sort find output in libtool for gcc-8 (Carl Dong) ee0a67c32a8861eab650bf8894af06807578eba0 codesigning: Use SHA256 as digest for osslsigncode (Windows) (Carl Dong) 38eb91eb0616ed6dbe34c23e11588d130fef07f8 guix: Add codesigning functionality (Carl Dong) bac2690e6f683fcedb883fe1d32f3c33c628a141 guix: Package codesigning tools (Carl Dong) 0a2176d47767972e4cd5ed302c1dbeedece1708b guix: Reindent existing manifest.scm (Carl Dong) c090a3e9238ba2df07875b4708e908d8dca4ed9b Makefile.am: use APP_DIST_DIR instead of hard-coding dist (Carl Dong) Pull request description: This is the last PR before we reach feature-parity with the Gitian process! Note: I tried using the `Makefile` inside the distsrc to make the dmg instead of manually listing out the commands, but `make` seems to want to re-make a lot of other files which broke the dmg. The workflow looks something like this: 1. `env [ FOO=bar... ] ./contrib/guix/guix-build` (add additional env vars as necessary) 2. Codesigners only: 1. Copy `guix-build-<short-id>/output/x86_64-apple-darwin18/bitcoin-<short-id>-osx-unsigned.tar.gz` and `guix-build-<short-id>/output/x86_64-w64-mingw32/bitcoin-<short-id>-win-unsigned.tar.gz` to signing computer 2. Codesign with `./detached-sig-create.sh` inside the tarball 3. Upload contents of `signature-{osx,win}.tar.gz` to https://github.com/bitcoin-core/bitcoin-detached-sigs (as a new tag) 3. Checkout new tag for `bitcoin-core/bitcoin-detached-sigs` with the detached signatures 4. `env [ FOO=bar... ] DETACHED_SIGS_REPO=<path/to/bitcoin-detached-sigs> ./contrib/guix/guix-codesign` (modify env vars as necessary) 5. Make sure `guix.sigs` is cloned and updated 6. `env GUIX_SIGS_REPO=<path/to/guix.sigs> SIGNER=0x96AB007F1A7ED999=dongcarl ./contrib/guix/guix-attest` (modify env vars as necessary) 7. Commit your new signatures and SHA256SUMS in `guix.sigs` 8. Optionally, after there are multiple signatures in `guix.sigs`: `env GUIX_SIGS_REPO=<path/to/guix.sigs> ./contrib/guix/guix-verify` ACKs for top commit: laanwj: Tested ACK ee883201cf134952284632e9e9ae72bf1c8c792f achow101: ACK ee883201cf134952284632e9e9ae72bf1c8c792f Tree-SHA512: e812a07a5f19f900600c70cb9c717769ef544a6c0c12760b5558b76b6b37df863257f3dbf38b0757e6e06e334470267e94c9f2bdbc27409d6837b1a0bfc6acbc
2021-05-24 15:31:05 +02:00
(inherit xgcc)
(arguments
(substitute-keyword-arguments (package-arguments xgcc)
((#:phases phases)
`(modify-phases ,phases
(add-after 'pre-configure 'replace-rpath-with-rpath-link
(lambda _
(substitute* (cons "gcc/config/rs6000/sysv4.h"
(find-files "gcc/config"
"^gnu-user.*\\.h$"))
(("-rpath=") "-rpath-link="))
#t))))))))
Merge #15277: contrib: Enable building in Guix containers 751549b52a9a4cd27389d807ae67f02bbb39cd7f contrib: guix: Additional clarifications re: substitutes (Carl Dong) cd3e947f50db7cfe05c05b368c25742193729a62 contrib: guix: Various improvements. (Carl Dong) 8dff3e48a9e03299468ed3b342642f01f70da9db contrib: guix: Clarify SOURCE_DATE_EPOCH. (Carl Dong) 3e80ec3ea9691c7c89173de922a113e643fe976b contrib: Add deterministic Guix builds. (Carl Dong) Pull request description: ~~**This post is kept updated as this project progresses. Use this [latest update link](https://github.com/bitcoin/bitcoin/pull/15277#issuecomment-497303718) to see what's new.**~~ Please read the `README.md`. ----- ### Guix Introduction This PR enables building bitcoin in Guix containers. [Guix](https://www.gnu.org/software/guix/manual/en/html_node/Features.html) is a transactional package manager much like Nix, but unlike Nix, it has more of a focus on [bootstrappability](https://www.gnu.org/software/guix/manual/en/html_node/Bootstrapping.html) and [reproducibility](https://www.gnu.org/software/guix/blog/tags/reproducible-builds/) which are attractive for security-sensitive projects like bitcoin. ### Guix Build Walkthrough Please read the `README.md`. [Old instructions no. 4](https://github.com/bitcoin/bitcoin/pull/15277#issuecomment-497303718) [Old instructions no. 3](https://github.com/bitcoin/bitcoin/pull/15277#issuecomment-493827011) [Old instructions no. 2](https://github.com/bitcoin/bitcoin/pull/15277#issuecomment-471658439) <details> <summary>Old instructions no. 1</summary> In this PR, we define a Guix [manifest](https://www.gnu.org/software/guix/manual/en/html_node/Invoking-guix-package.html#profile_002dmanifest) in `contrib/guix/manifest.scm`, which declares what packages we want in our environment. We can then invoke ``` guix environment --manifest=contrib/guix/manifest.scm --container --pure --no-grafts --no-substitutes ``` To have Guix: 1. Build an environment containing the packages we defined in our `contrib/guix/manifest.scm` manifest from the Guix bootstrap binaries (see [bootstrappability](https://www.gnu.org/software/guix/manual/en/html_node/Bootstrapping.html) for more details). 2. Start a container with that environment that has no network access, and no access to the host's filesystem except to the `pwd` that it was started in. 3. Drop you into a shell in that container. > Note: if you don't want to wait hours for Guix to build the entire world from scratch, you can eliminate the `--no-substitutes` option to have Guix download from available binary sources. Note that this convenience doesn't necessarily compromise your security, as you can check that a package was built correctly after the fact using `guix build --check <packagename>` Therefore, we can perform a build of bitcoin much like in Gitian by invoking the following: ``` make -C depends -j"$(nproc)" download && \ cat contrib/guix/build.sh | guix environment --manifest=contrib/guix/manifest.scm --container --pure --no-grafts --no-substitutes ``` We don't include `make -C depends -j"$(nproc)" download` inside `contrib/guix/build.sh` because `contrib/guix/build.sh` is run inside the container, which has no network access (which is a good thing). </details> ### Rationale I believe that this represents a substantial improvement for the "supply chain security" of bitcoin because: 1. We no longer have to rely on Ubuntu for our build environment for our releases ([oh the horror](https://github.com/bitcoin/bitcoin/blob/72bd4ab867e3be0d8410403d9641c08288d343e3/contrib/gitian-descriptors/gitian-linux.yml#L10)), because Guix builds everything about the container, we can perform this on almost any Linux distro/system. 2. It is now much easier to determine what trusted binaries are in our supply chain, and even make a nice visualization! (see [bootstrappability](https://www.gnu.org/software/guix/manual/en/html_node/Bootstrapping.html)). 3. There is active effort among Guix folks to minimize the number of trusted binaries even further. OriansJ's [stage0](https://github.com/oriansj/stage0), and janneke's [Mes](https://www.gnu.org/software/mes/) all aim to achieve [reduced binary boostrap](http://joyofsource.com/reduced-binary-seed-bootstrap.html) for Guix. In fact, I believe if OriansJ gets his way, we will end up some day with only a single trusted binary: hex0 (a ~500 byte self-hosting hex assembler). ### Steps to Completion - [x] Successfully build bitcoin inside the Guix environment - [x] Make `check-symbols` pass - [x] Do the above but without nasty hacks - [x] Solve some of the more innocuous hacks - [ ] Make it cross-compile (HELP WANTED HERE) - [x] Linux - [x] x86_64-linux-gnu - [x] i686-linux-gnu - [x] aarch64-linux-gnu - [x] arm-linux-gnueabihf - [x] riscv64-linux-gnu - [ ] OS X - [ ] x86_64-apple-darwin14 - [ ] Windows - [ ] x86_64-w64-mingw32 - [ ] Maybe make importer for depends syntax - [ ] Document build process for future releases - [ ] Extra: Pin the revision of Guix that we build with with Guix [inferiors](https://www.gnu.org/software/guix/manual/en/html_node/Inferiors.html) ### Help Wanted [Old content no. 3](https://github.com/bitcoin/bitcoin/pull/15277#issuecomment-483318210) [Old content no. 2](https://github.com/bitcoin/bitcoin/pull/15277#issuecomment-471658439) <details> <summary>Old content no. 1</summary> As of now, the command described above to perform a build of bitcoin a lot like Gitian works, but fails at the `check-symbols` stage. This is because a few dynamic libraries are linked in that shouldn't be. Here's what `ldd src/bitcoind` looks like when built in a Guix container: ``` linux-vdso.so.1 (0x00007ffcc2d90000) libdl.so.2 => /gnu/store/h90vnqw0nwd0hhm1l5dgxsdrigddfmq4-glibc-2.28/lib/libdl.so.2 (0x00007fb7eda09000) librt.so.1 => /gnu/store/h90vnqw0nwd0hhm1l5dgxsdrigddfmq4-glibc-2.28/lib/librt.so.1 (0x00007fb7ed9ff000) libstdc++.so.6 => /gnu/store/4sqps8dczv3g7rwbdibfz6rf5jlk7w90-gcc-5.5.0-lib/lib/libstdc++.so.6 (0x00007fb7ed87c000) libpthread.so.0 => /gnu/store/h90vnqw0nwd0hhm1l5dgxsdrigddfmq4-glibc-2.28/lib/libpthread.so.0 (0x00007fb7ed85b000) libm.so.6 => /gnu/store/h90vnqw0nwd0hhm1l5dgxsdrigddfmq4-glibc-2.28/lib/libm.so.6 (0x00007fb7ed6da000) libgcc_s.so.1 => /gnu/store/4sqps8dczv3g7rwbdibfz6rf5jlk7w90-gcc-5.5.0-lib/lib/libgcc_s.so.1 (0x00007fb7ed6bf000) libc.so.6 => /gnu/store/h90vnqw0nwd0hhm1l5dgxsdrigddfmq4-glibc-2.28/lib/libc.so.6 (0x00007fb7ed506000) /gnu/store/h90vnqw0nwd0hhm1l5dgxsdrigddfmq4-glibc-2.28/lib/ld-linux-x86-64.so.2 => /usr/lib64/ld-linux-x86-64.so.2 (0x00007fb7ee3a0000) ``` And here's what it looks in one of our releases: ``` linux-vdso.so.1 (0x00007ffff52cd000) libpthread.so.0 => /usr/lib/libpthread.so.0 (0x00007f87726b4000) librt.so.1 => /usr/lib/librt.so.1 (0x00007f87726aa000) libm.so.6 => /usr/lib/libm.so.6 (0x00007f8772525000) libgcc_s.so.1 => /usr/lib/libgcc_s.so.1 (0x00007f877250b000) libc.so.6 => /usr/lib/libc.so.6 (0x00007f8772347000) /lib64/ld-linux-x86-64.so.2 => /usr/lib64/ld-linux-x86-64.so.2 (0x00007f8773392000) ``` ~~I suspect it is because my script does not apply the gitian-input patches [described in the release process](https://github.com/bitcoin/bitcoin/blob/master/doc/release-process.md#fetch-and-create-inputs-first-time-or-when-dependency-versions-change) but there is no description as to how these patches are applied.~~ It might also be something else entirely. Edit: It is something else. It appears that the gitian inputs are only used by [`gitian-win-signer.yml`](https://github.com/bitcoin/bitcoin/blob/d6e700e40f861ddd6743f4d13f0d6f6bc19093c2/contrib/gitian-descriptors/gitian-win-signer.yml#L14) </details> ### How to Help 1. Install Guix on your distro either [from source](https://www.gnu.org/software/guix/manual/en/html_node/Requirements.html) or perform a [binary installation](https://www.gnu.org/software/guix/manual/en/html_node/Binary-Installation.html#Binary-Installation) 2. Try out my branch and the command described above! ACKs for top commit: MarcoFalke: Thanks for the replies. ACK 751549b52a9a4cd27389d807ae67f02bbb39cd7f laanwj: ACK 751549b52a9a4cd27389d807ae67f02bbb39cd7f Tree-SHA512: 50e6ab58c6bda9a67125b6271daf7eff0ca57d0efa8941ed3cd951e5bf78b31552fc5e537b1e1bcf2d3cc918c63adf19d685aa117a0f851024dc67e697890a8d
2019-07-12 19:23:30 +02:00
Merge bitcoin/bitcoin#25861: guix: use --build={arch}-guix-linux-gnu in cross toolchain 56e79fe683d36c1944e52326fae3bcc4cb7deec7 guix: use --build={arch}-guix-linux-gnu in cross toolchain (fanquake) Pull request description: Technically we are always cross-compiling when Guix building, so make that explicit. `{arch}-guix-linux-gnu` is not a triplet that should be used in any other capacity, but here it serves the purpose of ensuring, that by setting `--build` to something other than `--host/--target`, we are always cross-compiling (in the eyes of autoconf etc) when building our cross toolchains. It looks like `x86_64-linux-gnu` on `x86_64-linux-gnu` currently works because of the triplet canonicalisation, i.e `x86_64-linux-gnu` becomes `x86_64-pc-linux-gnu`, and GCCs configure thinking it's cross-compiling, whereas the same canonicalisation doesn't happen for `aarch64-linux-gnu` so we don't get a cross-compile when building on aarch64. Fixes: #22458. Guix Build (x86_64): ```bash e590e67d2b151e7d079f50393c61cacf9e65f5e3fdddc10bffd5dbb59cd9f5e3 guix-build-56e79fe683d3/output/aarch64-linux-gnu/SHA256SUMS.part 9222a733209dc73168722ee0f290831d1723f7ed55abc1e243dd3892ef9733f1 guix-build-56e79fe683d3/output/aarch64-linux-gnu/bitcoin-56e79fe683d3-aarch64-linux-gnu-debug.tar.gz a4ec01411195412e9a483397554d98b2db161a6be8aceee2dac78f0015f2a8e5 guix-build-56e79fe683d3/output/aarch64-linux-gnu/bitcoin-56e79fe683d3-aarch64-linux-gnu.tar.gz a4ff8f2538c1b3a63a3e7c92699c18164b2c4954ebd72c4a854bf7844e6131b6 guix-build-56e79fe683d3/output/arm-linux-gnueabihf/SHA256SUMS.part ac6e4dffcf6f49dac73d3677f3fe5edb08ee1127648bc7ec24b217f48a3a4d21 guix-build-56e79fe683d3/output/arm-linux-gnueabihf/bitcoin-56e79fe683d3-arm-linux-gnueabihf-debug.tar.gz ab26467fce9732596b1713179618eee20c2bd44875dc1c39f9afd68f135e8ce4 guix-build-56e79fe683d3/output/arm-linux-gnueabihf/bitcoin-56e79fe683d3-arm-linux-gnueabihf.tar.gz f44108d818d53ee7e70b44361cb3cc6841b7aecbe4fade1fa289263718620824 guix-build-56e79fe683d3/output/arm64-apple-darwin/SHA256SUMS.part 29adfb2d575be8b5a7092b2aff888af2084ad5783bdc74ae3de4b2dd84da7a26 guix-build-56e79fe683d3/output/arm64-apple-darwin/bitcoin-56e79fe683d3-arm64-apple-darwin-unsigned.dmg 3d7640f604bd27cb1ae47853d70378a7631aa436b5c027eae31fa06d4dd20d4a guix-build-56e79fe683d3/output/arm64-apple-darwin/bitcoin-56e79fe683d3-arm64-apple-darwin-unsigned.tar.gz 3f8d8fc49cf722a7a13640872762d045c14efd610fba908f0acc99a99032b903 guix-build-56e79fe683d3/output/arm64-apple-darwin/bitcoin-56e79fe683d3-arm64-apple-darwin.tar.gz b8df097e10e7acded8556a5681da2b0132c0d509b200792f5584c186b8dcaf24 guix-build-56e79fe683d3/output/dist-archive/bitcoin-56e79fe683d3.tar.gz 1a605c563038107b88e8d859fa045ca3087241340264eb7125e97b3af36874db guix-build-56e79fe683d3/output/powerpc64-linux-gnu/SHA256SUMS.part d10be89a816ee2f47bf778ef4aaa8a7b548403c123a4b4b3aa0470ee7b9472d2 guix-build-56e79fe683d3/output/powerpc64-linux-gnu/bitcoin-56e79fe683d3-powerpc64-linux-gnu-debug.tar.gz 839dd079d677c87a91ccc28f6c423657d98212350d3693ae2facb92082d10c88 guix-build-56e79fe683d3/output/powerpc64-linux-gnu/bitcoin-56e79fe683d3-powerpc64-linux-gnu.tar.gz 200d7a72a7901b6788e70aa1c6fbbe16144e9dac29beb06242a9dcfe1e88acbe guix-build-56e79fe683d3/output/powerpc64le-linux-gnu/SHA256SUMS.part 3d207bb57dc45f234d804a9ee615001aa28ac4b59b530a1e7d16d0a72c2ea514 guix-build-56e79fe683d3/output/powerpc64le-linux-gnu/bitcoin-56e79fe683d3-powerpc64le-linux-gnu-debug.tar.gz 16a789c98409d36bd6b69a5ca38cd8886979f50f845265dc0b005c92286a3e4f guix-build-56e79fe683d3/output/powerpc64le-linux-gnu/bitcoin-56e79fe683d3-powerpc64le-linux-gnu.tar.gz e48f6fae9c2ffeb2bf02ffc073ac69648b2ce4ba87e0ad7a3a5059abf33b8ce2 guix-build-56e79fe683d3/output/riscv64-linux-gnu/SHA256SUMS.part fa8d66af5f202e0c8968114af4dcd6d88099eb91d197dfaaa0144f8ae6b12f4a guix-build-56e79fe683d3/output/riscv64-linux-gnu/bitcoin-56e79fe683d3-riscv64-linux-gnu-debug.tar.gz 86cffa62687f4272c899897ff6c1a86e1a7bdbbb7ab09a085b7b1122d3a1f1b2 guix-build-56e79fe683d3/output/riscv64-linux-gnu/bitcoin-56e79fe683d3-riscv64-linux-gnu.tar.gz e42a7c33da6ea4be5d0826df097bdc7e919c9fd6642e7d59fe3d45a97d9009fa guix-build-56e79fe683d3/output/x86_64-apple-darwin/SHA256SUMS.part 2f9faa88fa0025b9bfade83177b1f50833dc6c3be1ef26c4d5881aa165bcc537 guix-build-56e79fe683d3/output/x86_64-apple-darwin/bitcoin-56e79fe683d3-x86_64-apple-darwin-unsigned.dmg fdfe560968c85500dc886d4e12b851cd27a76de860b8cbf1fb6aac95ccd0582e guix-build-56e79fe683d3/output/x86_64-apple-darwin/bitcoin-56e79fe683d3-x86_64-apple-darwin-unsigned.tar.gz c3c75560befd72a9f13f5626ddf39c63ad25fb9753a4d941e48830b7e06f143e guix-build-56e79fe683d3/output/x86_64-apple-darwin/bitcoin-56e79fe683d3-x86_64-apple-darwin.tar.gz 38100946ec1469fdd31aa8ea9e471b317dc278c298c106289c119016c2c0caa0 guix-build-56e79fe683d3/output/x86_64-linux-gnu/SHA256SUMS.part 67890771aad89dc0d20b8dcdc6b29bd838ace310e8bcbbc0d16d7afc87d3a474 guix-build-56e79fe683d3/output/x86_64-linux-gnu/bitcoin-56e79fe683d3-x86_64-linux-gnu-debug.tar.gz a9772b40c8b9d3c1a0bf9c9db2a60050415bf6a9bd4a19cfd8a038f4a1e712e3 guix-build-56e79fe683d3/output/x86_64-linux-gnu/bitcoin-56e79fe683d3-x86_64-linux-gnu.tar.gz 01da0561f2c3a268c28615b1573dfdecb6346bf3f8eba7a052488191234ac1d6 guix-build-56e79fe683d3/output/x86_64-w64-mingw32/SHA256SUMS.part 71324ac0f6fc8163e41031b2fd2d38d2e15bfd5ef4efe4cb45d7974fad474394 guix-build-56e79fe683d3/output/x86_64-w64-mingw32/bitcoin-56e79fe683d3-win64-debug.zip 219b58281d3c79b2ed7e9085b1e15d7e021fd3899ef07a6ad747058b43d64443 guix-build-56e79fe683d3/output/x86_64-w64-mingw32/bitcoin-56e79fe683d3-win64-setup-unsigned.exe fe838d32587c2f942fcd9f9a0b3735611b686a867e2d2040cf3a8aa6a43d09fe guix-build-56e79fe683d3/output/x86_64-w64-mingw32/bitcoin-56e79fe683d3-win64-unsigned.tar.gz 13b537217a13293b93afd7c588b3733955c3188f79a3249b363fb4e885b74b32 guix-build-56e79fe683d3/output/x86_64-w64-mingw32/bitcoin-56e79fe683d3-win64.zip ``` Guix Build (arm64): ```bash e590e67d2b151e7d079f50393c61cacf9e65f5e3fdddc10bffd5dbb59cd9f5e3 guix-build-56e79fe683d3/output/aarch64-linux-gnu/SHA256SUMS.part 9222a733209dc73168722ee0f290831d1723f7ed55abc1e243dd3892ef9733f1 guix-build-56e79fe683d3/output/aarch64-linux-gnu/bitcoin-56e79fe683d3-aarch64-linux-gnu-debug.tar.gz a4ec01411195412e9a483397554d98b2db161a6be8aceee2dac78f0015f2a8e5 guix-build-56e79fe683d3/output/aarch64-linux-gnu/bitcoin-56e79fe683d3-aarch64-linux-gnu.tar.gz a4ff8f2538c1b3a63a3e7c92699c18164b2c4954ebd72c4a854bf7844e6131b6 guix-build-56e79fe683d3/output/arm-linux-gnueabihf/SHA256SUMS.part ac6e4dffcf6f49dac73d3677f3fe5edb08ee1127648bc7ec24b217f48a3a4d21 guix-build-56e79fe683d3/output/arm-linux-gnueabihf/bitcoin-56e79fe683d3-arm-linux-gnueabihf-debug.tar.gz ab26467fce9732596b1713179618eee20c2bd44875dc1c39f9afd68f135e8ce4 guix-build-56e79fe683d3/output/arm-linux-gnueabihf/bitcoin-56e79fe683d3-arm-linux-gnueabihf.tar.gz f44108d818d53ee7e70b44361cb3cc6841b7aecbe4fade1fa289263718620824 guix-build-56e79fe683d3/output/arm64-apple-darwin/SHA256SUMS.part 29adfb2d575be8b5a7092b2aff888af2084ad5783bdc74ae3de4b2dd84da7a26 guix-build-56e79fe683d3/output/arm64-apple-darwin/bitcoin-56e79fe683d3-arm64-apple-darwin-unsigned.dmg 3d7640f604bd27cb1ae47853d70378a7631aa436b5c027eae31fa06d4dd20d4a guix-build-56e79fe683d3/output/arm64-apple-darwin/bitcoin-56e79fe683d3-arm64-apple-darwin-unsigned.tar.gz 3f8d8fc49cf722a7a13640872762d045c14efd610fba908f0acc99a99032b903 guix-build-56e79fe683d3/output/arm64-apple-darwin/bitcoin-56e79fe683d3-arm64-apple-darwin.tar.gz b8df097e10e7acded8556a5681da2b0132c0d509b200792f5584c186b8dcaf24 guix-build-56e79fe683d3/output/dist-archive/bitcoin-56e79fe683d3.tar.gz 1a605c563038107b88e8d859fa045ca3087241340264eb7125e97b3af36874db guix-build-56e79fe683d3/output/powerpc64-linux-gnu/SHA256SUMS.part d10be89a816ee2f47bf778ef4aaa8a7b548403c123a4b4b3aa0470ee7b9472d2 guix-build-56e79fe683d3/output/powerpc64-linux-gnu/bitcoin-56e79fe683d3-powerpc64-linux-gnu-debug.tar.gz 839dd079d677c87a91ccc28f6c423657d98212350d3693ae2facb92082d10c88 guix-build-56e79fe683d3/output/powerpc64-linux-gnu/bitcoin-56e79fe683d3-powerpc64-linux-gnu.tar.gz 200d7a72a7901b6788e70aa1c6fbbe16144e9dac29beb06242a9dcfe1e88acbe guix-build-56e79fe683d3/output/powerpc64le-linux-gnu/SHA256SUMS.part 3d207bb57dc45f234d804a9ee615001aa28ac4b59b530a1e7d16d0a72c2ea514 guix-build-56e79fe683d3/output/powerpc64le-linux-gnu/bitcoin-56e79fe683d3-powerpc64le-linux-gnu-debug.tar.gz 16a789c98409d36bd6b69a5ca38cd8886979f50f845265dc0b005c92286a3e4f guix-build-56e79fe683d3/output/powerpc64le-linux-gnu/bitcoin-56e79fe683d3-powerpc64le-linux-gnu.tar.gz e48f6fae9c2ffeb2bf02ffc073ac69648b2ce4ba87e0ad7a3a5059abf33b8ce2 guix-build-56e79fe683d3/output/riscv64-linux-gnu/SHA256SUMS.part fa8d66af5f202e0c8968114af4dcd6d88099eb91d197dfaaa0144f8ae6b12f4a guix-build-56e79fe683d3/output/riscv64-linux-gnu/bitcoin-56e79fe683d3-riscv64-linux-gnu-debug.tar.gz 86cffa62687f4272c899897ff6c1a86e1a7bdbbb7ab09a085b7b1122d3a1f1b2 guix-build-56e79fe683d3/output/riscv64-linux-gnu/bitcoin-56e79fe683d3-riscv64-linux-gnu.tar.gz e42a7c33da6ea4be5d0826df097bdc7e919c9fd6642e7d59fe3d45a97d9009fa guix-build-56e79fe683d3/output/x86_64-apple-darwin/SHA256SUMS.part 2f9faa88fa0025b9bfade83177b1f50833dc6c3be1ef26c4d5881aa165bcc537 guix-build-56e79fe683d3/output/x86_64-apple-darwin/bitcoin-56e79fe683d3-x86_64-apple-darwin-unsigned.dmg fdfe560968c85500dc886d4e12b851cd27a76de860b8cbf1fb6aac95ccd0582e guix-build-56e79fe683d3/output/x86_64-apple-darwin/bitcoin-56e79fe683d3-x86_64-apple-darwin-unsigned.tar.gz c3c75560befd72a9f13f5626ddf39c63ad25fb9753a4d941e48830b7e06f143e guix-build-56e79fe683d3/output/x86_64-apple-darwin/bitcoin-56e79fe683d3-x86_64-apple-darwin.tar.gz 38100946ec1469fdd31aa8ea9e471b317dc278c298c106289c119016c2c0caa0 guix-build-56e79fe683d3/output/x86_64-linux-gnu/SHA256SUMS.part 67890771aad89dc0d20b8dcdc6b29bd838ace310e8bcbbc0d16d7afc87d3a474 guix-build-56e79fe683d3/output/x86_64-linux-gnu/bitcoin-56e79fe683d3-x86_64-linux-gnu-debug.tar.gz a9772b40c8b9d3c1a0bf9c9db2a60050415bf6a9bd4a19cfd8a038f4a1e712e3 guix-build-56e79fe683d3/output/x86_64-linux-gnu/bitcoin-56e79fe683d3-x86_64-linux-gnu.tar.gz 01da0561f2c3a268c28615b1573dfdecb6346bf3f8eba7a052488191234ac1d6 guix-build-56e79fe683d3/output/x86_64-w64-mingw32/SHA256SUMS.part 71324ac0f6fc8163e41031b2fd2d38d2e15bfd5ef4efe4cb45d7974fad474394 guix-build-56e79fe683d3/output/x86_64-w64-mingw32/bitcoin-56e79fe683d3-win64-debug.zip 219b58281d3c79b2ed7e9085b1e15d7e021fd3899ef07a6ad747058b43d64443 guix-build-56e79fe683d3/output/x86_64-w64-mingw32/bitcoin-56e79fe683d3-win64-setup-unsigned.exe fe838d32587c2f942fcd9f9a0b3735611b686a867e2d2040cf3a8aa6a43d09fe guix-build-56e79fe683d3/output/x86_64-w64-mingw32/bitcoin-56e79fe683d3-win64-unsigned.tar.gz 13b537217a13293b93afd7c588b3733955c3188f79a3249b363fb4e885b74b32 guix-build-56e79fe683d3/output/x86_64-w64-mingw32/bitcoin-56e79fe683d3-win64.zip ``` ACKs for top commit: hebasto: ACK 56e79fe683d36c1944e52326fae3bcc4cb7deec7 Tree-SHA512: 628ab6cda80069ad277107639bef21b44a8417198862e9ec89b45a2c41741d29aeb79aa58c5a90283fb96cf707494ae948ac790abde809bb18c86b14af999200
2022-09-04 19:20:16 +02:00
(define building-on (string-append (list-ref (string-split (%current-system) #\-) 0) "-guix-linux-gnu"))
(define (explicit-cross-configure package)
(package-with-extra-configure-variable package "--build" building-on))
(define (make-binutils-with-mingw-w64-disable-flags xbinutils)
(package-with-extra-patches xbinutils
(search-our-patches "binutils-mingw-w64-disable-flags.patch")))
Merge #15277: contrib: Enable building in Guix containers 751549b52a9a4cd27389d807ae67f02bbb39cd7f contrib: guix: Additional clarifications re: substitutes (Carl Dong) cd3e947f50db7cfe05c05b368c25742193729a62 contrib: guix: Various improvements. (Carl Dong) 8dff3e48a9e03299468ed3b342642f01f70da9db contrib: guix: Clarify SOURCE_DATE_EPOCH. (Carl Dong) 3e80ec3ea9691c7c89173de922a113e643fe976b contrib: Add deterministic Guix builds. (Carl Dong) Pull request description: ~~**This post is kept updated as this project progresses. Use this [latest update link](https://github.com/bitcoin/bitcoin/pull/15277#issuecomment-497303718) to see what's new.**~~ Please read the `README.md`. ----- ### Guix Introduction This PR enables building bitcoin in Guix containers. [Guix](https://www.gnu.org/software/guix/manual/en/html_node/Features.html) is a transactional package manager much like Nix, but unlike Nix, it has more of a focus on [bootstrappability](https://www.gnu.org/software/guix/manual/en/html_node/Bootstrapping.html) and [reproducibility](https://www.gnu.org/software/guix/blog/tags/reproducible-builds/) which are attractive for security-sensitive projects like bitcoin. ### Guix Build Walkthrough Please read the `README.md`. [Old instructions no. 4](https://github.com/bitcoin/bitcoin/pull/15277#issuecomment-497303718) [Old instructions no. 3](https://github.com/bitcoin/bitcoin/pull/15277#issuecomment-493827011) [Old instructions no. 2](https://github.com/bitcoin/bitcoin/pull/15277#issuecomment-471658439) <details> <summary>Old instructions no. 1</summary> In this PR, we define a Guix [manifest](https://www.gnu.org/software/guix/manual/en/html_node/Invoking-guix-package.html#profile_002dmanifest) in `contrib/guix/manifest.scm`, which declares what packages we want in our environment. We can then invoke ``` guix environment --manifest=contrib/guix/manifest.scm --container --pure --no-grafts --no-substitutes ``` To have Guix: 1. Build an environment containing the packages we defined in our `contrib/guix/manifest.scm` manifest from the Guix bootstrap binaries (see [bootstrappability](https://www.gnu.org/software/guix/manual/en/html_node/Bootstrapping.html) for more details). 2. Start a container with that environment that has no network access, and no access to the host's filesystem except to the `pwd` that it was started in. 3. Drop you into a shell in that container. > Note: if you don't want to wait hours for Guix to build the entire world from scratch, you can eliminate the `--no-substitutes` option to have Guix download from available binary sources. Note that this convenience doesn't necessarily compromise your security, as you can check that a package was built correctly after the fact using `guix build --check <packagename>` Therefore, we can perform a build of bitcoin much like in Gitian by invoking the following: ``` make -C depends -j"$(nproc)" download && \ cat contrib/guix/build.sh | guix environment --manifest=contrib/guix/manifest.scm --container --pure --no-grafts --no-substitutes ``` We don't include `make -C depends -j"$(nproc)" download` inside `contrib/guix/build.sh` because `contrib/guix/build.sh` is run inside the container, which has no network access (which is a good thing). </details> ### Rationale I believe that this represents a substantial improvement for the "supply chain security" of bitcoin because: 1. We no longer have to rely on Ubuntu for our build environment for our releases ([oh the horror](https://github.com/bitcoin/bitcoin/blob/72bd4ab867e3be0d8410403d9641c08288d343e3/contrib/gitian-descriptors/gitian-linux.yml#L10)), because Guix builds everything about the container, we can perform this on almost any Linux distro/system. 2. It is now much easier to determine what trusted binaries are in our supply chain, and even make a nice visualization! (see [bootstrappability](https://www.gnu.org/software/guix/manual/en/html_node/Bootstrapping.html)). 3. There is active effort among Guix folks to minimize the number of trusted binaries even further. OriansJ's [stage0](https://github.com/oriansj/stage0), and janneke's [Mes](https://www.gnu.org/software/mes/) all aim to achieve [reduced binary boostrap](http://joyofsource.com/reduced-binary-seed-bootstrap.html) for Guix. In fact, I believe if OriansJ gets his way, we will end up some day with only a single trusted binary: hex0 (a ~500 byte self-hosting hex assembler). ### Steps to Completion - [x] Successfully build bitcoin inside the Guix environment - [x] Make `check-symbols` pass - [x] Do the above but without nasty hacks - [x] Solve some of the more innocuous hacks - [ ] Make it cross-compile (HELP WANTED HERE) - [x] Linux - [x] x86_64-linux-gnu - [x] i686-linux-gnu - [x] aarch64-linux-gnu - [x] arm-linux-gnueabihf - [x] riscv64-linux-gnu - [ ] OS X - [ ] x86_64-apple-darwin14 - [ ] Windows - [ ] x86_64-w64-mingw32 - [ ] Maybe make importer for depends syntax - [ ] Document build process for future releases - [ ] Extra: Pin the revision of Guix that we build with with Guix [inferiors](https://www.gnu.org/software/guix/manual/en/html_node/Inferiors.html) ### Help Wanted [Old content no. 3](https://github.com/bitcoin/bitcoin/pull/15277#issuecomment-483318210) [Old content no. 2](https://github.com/bitcoin/bitcoin/pull/15277#issuecomment-471658439) <details> <summary>Old content no. 1</summary> As of now, the command described above to perform a build of bitcoin a lot like Gitian works, but fails at the `check-symbols` stage. This is because a few dynamic libraries are linked in that shouldn't be. Here's what `ldd src/bitcoind` looks like when built in a Guix container: ``` linux-vdso.so.1 (0x00007ffcc2d90000) libdl.so.2 => /gnu/store/h90vnqw0nwd0hhm1l5dgxsdrigddfmq4-glibc-2.28/lib/libdl.so.2 (0x00007fb7eda09000) librt.so.1 => /gnu/store/h90vnqw0nwd0hhm1l5dgxsdrigddfmq4-glibc-2.28/lib/librt.so.1 (0x00007fb7ed9ff000) libstdc++.so.6 => /gnu/store/4sqps8dczv3g7rwbdibfz6rf5jlk7w90-gcc-5.5.0-lib/lib/libstdc++.so.6 (0x00007fb7ed87c000) libpthread.so.0 => /gnu/store/h90vnqw0nwd0hhm1l5dgxsdrigddfmq4-glibc-2.28/lib/libpthread.so.0 (0x00007fb7ed85b000) libm.so.6 => /gnu/store/h90vnqw0nwd0hhm1l5dgxsdrigddfmq4-glibc-2.28/lib/libm.so.6 (0x00007fb7ed6da000) libgcc_s.so.1 => /gnu/store/4sqps8dczv3g7rwbdibfz6rf5jlk7w90-gcc-5.5.0-lib/lib/libgcc_s.so.1 (0x00007fb7ed6bf000) libc.so.6 => /gnu/store/h90vnqw0nwd0hhm1l5dgxsdrigddfmq4-glibc-2.28/lib/libc.so.6 (0x00007fb7ed506000) /gnu/store/h90vnqw0nwd0hhm1l5dgxsdrigddfmq4-glibc-2.28/lib/ld-linux-x86-64.so.2 => /usr/lib64/ld-linux-x86-64.so.2 (0x00007fb7ee3a0000) ``` And here's what it looks in one of our releases: ``` linux-vdso.so.1 (0x00007ffff52cd000) libpthread.so.0 => /usr/lib/libpthread.so.0 (0x00007f87726b4000) librt.so.1 => /usr/lib/librt.so.1 (0x00007f87726aa000) libm.so.6 => /usr/lib/libm.so.6 (0x00007f8772525000) libgcc_s.so.1 => /usr/lib/libgcc_s.so.1 (0x00007f877250b000) libc.so.6 => /usr/lib/libc.so.6 (0x00007f8772347000) /lib64/ld-linux-x86-64.so.2 => /usr/lib64/ld-linux-x86-64.so.2 (0x00007f8773392000) ``` ~~I suspect it is because my script does not apply the gitian-input patches [described in the release process](https://github.com/bitcoin/bitcoin/blob/master/doc/release-process.md#fetch-and-create-inputs-first-time-or-when-dependency-versions-change) but there is no description as to how these patches are applied.~~ It might also be something else entirely. Edit: It is something else. It appears that the gitian inputs are only used by [`gitian-win-signer.yml`](https://github.com/bitcoin/bitcoin/blob/d6e700e40f861ddd6743f4d13f0d6f6bc19093c2/contrib/gitian-descriptors/gitian-win-signer.yml#L14) </details> ### How to Help 1. Install Guix on your distro either [from source](https://www.gnu.org/software/guix/manual/en/html_node/Requirements.html) or perform a [binary installation](https://www.gnu.org/software/guix/manual/en/html_node/Binary-Installation.html#Binary-Installation) 2. Try out my branch and the command described above! ACKs for top commit: MarcoFalke: Thanks for the replies. ACK 751549b52a9a4cd27389d807ae67f02bbb39cd7f laanwj: ACK 751549b52a9a4cd27389d807ae67f02bbb39cd7f Tree-SHA512: 50e6ab58c6bda9a67125b6271daf7eff0ca57d0efa8941ed3cd951e5bf78b31552fc5e537b1e1bcf2d3cc918c63adf19d685aa117a0f851024dc67e697890a8d
2019-07-12 19:23:30 +02:00
(define (make-cross-toolchain target
base-gcc-for-libc
base-kernel-headers
base-libc
base-gcc)
"Create a cross-compilation toolchain package for TARGET"
(let* ((xbinutils (cross-binutils target))
;; 1. Build a cross-compiling gcc without targeting any libc, derived
;; from BASE-GCC-FOR-LIBC
Merge bitcoin/bitcoin#25861: guix: use --build={arch}-guix-linux-gnu in cross toolchain 56e79fe683d36c1944e52326fae3bcc4cb7deec7 guix: use --build={arch}-guix-linux-gnu in cross toolchain (fanquake) Pull request description: Technically we are always cross-compiling when Guix building, so make that explicit. `{arch}-guix-linux-gnu` is not a triplet that should be used in any other capacity, but here it serves the purpose of ensuring, that by setting `--build` to something other than `--host/--target`, we are always cross-compiling (in the eyes of autoconf etc) when building our cross toolchains. It looks like `x86_64-linux-gnu` on `x86_64-linux-gnu` currently works because of the triplet canonicalisation, i.e `x86_64-linux-gnu` becomes `x86_64-pc-linux-gnu`, and GCCs configure thinking it's cross-compiling, whereas the same canonicalisation doesn't happen for `aarch64-linux-gnu` so we don't get a cross-compile when building on aarch64. Fixes: #22458. Guix Build (x86_64): ```bash e590e67d2b151e7d079f50393c61cacf9e65f5e3fdddc10bffd5dbb59cd9f5e3 guix-build-56e79fe683d3/output/aarch64-linux-gnu/SHA256SUMS.part 9222a733209dc73168722ee0f290831d1723f7ed55abc1e243dd3892ef9733f1 guix-build-56e79fe683d3/output/aarch64-linux-gnu/bitcoin-56e79fe683d3-aarch64-linux-gnu-debug.tar.gz a4ec01411195412e9a483397554d98b2db161a6be8aceee2dac78f0015f2a8e5 guix-build-56e79fe683d3/output/aarch64-linux-gnu/bitcoin-56e79fe683d3-aarch64-linux-gnu.tar.gz a4ff8f2538c1b3a63a3e7c92699c18164b2c4954ebd72c4a854bf7844e6131b6 guix-build-56e79fe683d3/output/arm-linux-gnueabihf/SHA256SUMS.part ac6e4dffcf6f49dac73d3677f3fe5edb08ee1127648bc7ec24b217f48a3a4d21 guix-build-56e79fe683d3/output/arm-linux-gnueabihf/bitcoin-56e79fe683d3-arm-linux-gnueabihf-debug.tar.gz ab26467fce9732596b1713179618eee20c2bd44875dc1c39f9afd68f135e8ce4 guix-build-56e79fe683d3/output/arm-linux-gnueabihf/bitcoin-56e79fe683d3-arm-linux-gnueabihf.tar.gz f44108d818d53ee7e70b44361cb3cc6841b7aecbe4fade1fa289263718620824 guix-build-56e79fe683d3/output/arm64-apple-darwin/SHA256SUMS.part 29adfb2d575be8b5a7092b2aff888af2084ad5783bdc74ae3de4b2dd84da7a26 guix-build-56e79fe683d3/output/arm64-apple-darwin/bitcoin-56e79fe683d3-arm64-apple-darwin-unsigned.dmg 3d7640f604bd27cb1ae47853d70378a7631aa436b5c027eae31fa06d4dd20d4a guix-build-56e79fe683d3/output/arm64-apple-darwin/bitcoin-56e79fe683d3-arm64-apple-darwin-unsigned.tar.gz 3f8d8fc49cf722a7a13640872762d045c14efd610fba908f0acc99a99032b903 guix-build-56e79fe683d3/output/arm64-apple-darwin/bitcoin-56e79fe683d3-arm64-apple-darwin.tar.gz b8df097e10e7acded8556a5681da2b0132c0d509b200792f5584c186b8dcaf24 guix-build-56e79fe683d3/output/dist-archive/bitcoin-56e79fe683d3.tar.gz 1a605c563038107b88e8d859fa045ca3087241340264eb7125e97b3af36874db guix-build-56e79fe683d3/output/powerpc64-linux-gnu/SHA256SUMS.part d10be89a816ee2f47bf778ef4aaa8a7b548403c123a4b4b3aa0470ee7b9472d2 guix-build-56e79fe683d3/output/powerpc64-linux-gnu/bitcoin-56e79fe683d3-powerpc64-linux-gnu-debug.tar.gz 839dd079d677c87a91ccc28f6c423657d98212350d3693ae2facb92082d10c88 guix-build-56e79fe683d3/output/powerpc64-linux-gnu/bitcoin-56e79fe683d3-powerpc64-linux-gnu.tar.gz 200d7a72a7901b6788e70aa1c6fbbe16144e9dac29beb06242a9dcfe1e88acbe guix-build-56e79fe683d3/output/powerpc64le-linux-gnu/SHA256SUMS.part 3d207bb57dc45f234d804a9ee615001aa28ac4b59b530a1e7d16d0a72c2ea514 guix-build-56e79fe683d3/output/powerpc64le-linux-gnu/bitcoin-56e79fe683d3-powerpc64le-linux-gnu-debug.tar.gz 16a789c98409d36bd6b69a5ca38cd8886979f50f845265dc0b005c92286a3e4f guix-build-56e79fe683d3/output/powerpc64le-linux-gnu/bitcoin-56e79fe683d3-powerpc64le-linux-gnu.tar.gz e48f6fae9c2ffeb2bf02ffc073ac69648b2ce4ba87e0ad7a3a5059abf33b8ce2 guix-build-56e79fe683d3/output/riscv64-linux-gnu/SHA256SUMS.part fa8d66af5f202e0c8968114af4dcd6d88099eb91d197dfaaa0144f8ae6b12f4a guix-build-56e79fe683d3/output/riscv64-linux-gnu/bitcoin-56e79fe683d3-riscv64-linux-gnu-debug.tar.gz 86cffa62687f4272c899897ff6c1a86e1a7bdbbb7ab09a085b7b1122d3a1f1b2 guix-build-56e79fe683d3/output/riscv64-linux-gnu/bitcoin-56e79fe683d3-riscv64-linux-gnu.tar.gz e42a7c33da6ea4be5d0826df097bdc7e919c9fd6642e7d59fe3d45a97d9009fa guix-build-56e79fe683d3/output/x86_64-apple-darwin/SHA256SUMS.part 2f9faa88fa0025b9bfade83177b1f50833dc6c3be1ef26c4d5881aa165bcc537 guix-build-56e79fe683d3/output/x86_64-apple-darwin/bitcoin-56e79fe683d3-x86_64-apple-darwin-unsigned.dmg fdfe560968c85500dc886d4e12b851cd27a76de860b8cbf1fb6aac95ccd0582e guix-build-56e79fe683d3/output/x86_64-apple-darwin/bitcoin-56e79fe683d3-x86_64-apple-darwin-unsigned.tar.gz c3c75560befd72a9f13f5626ddf39c63ad25fb9753a4d941e48830b7e06f143e guix-build-56e79fe683d3/output/x86_64-apple-darwin/bitcoin-56e79fe683d3-x86_64-apple-darwin.tar.gz 38100946ec1469fdd31aa8ea9e471b317dc278c298c106289c119016c2c0caa0 guix-build-56e79fe683d3/output/x86_64-linux-gnu/SHA256SUMS.part 67890771aad89dc0d20b8dcdc6b29bd838ace310e8bcbbc0d16d7afc87d3a474 guix-build-56e79fe683d3/output/x86_64-linux-gnu/bitcoin-56e79fe683d3-x86_64-linux-gnu-debug.tar.gz a9772b40c8b9d3c1a0bf9c9db2a60050415bf6a9bd4a19cfd8a038f4a1e712e3 guix-build-56e79fe683d3/output/x86_64-linux-gnu/bitcoin-56e79fe683d3-x86_64-linux-gnu.tar.gz 01da0561f2c3a268c28615b1573dfdecb6346bf3f8eba7a052488191234ac1d6 guix-build-56e79fe683d3/output/x86_64-w64-mingw32/SHA256SUMS.part 71324ac0f6fc8163e41031b2fd2d38d2e15bfd5ef4efe4cb45d7974fad474394 guix-build-56e79fe683d3/output/x86_64-w64-mingw32/bitcoin-56e79fe683d3-win64-debug.zip 219b58281d3c79b2ed7e9085b1e15d7e021fd3899ef07a6ad747058b43d64443 guix-build-56e79fe683d3/output/x86_64-w64-mingw32/bitcoin-56e79fe683d3-win64-setup-unsigned.exe fe838d32587c2f942fcd9f9a0b3735611b686a867e2d2040cf3a8aa6a43d09fe guix-build-56e79fe683d3/output/x86_64-w64-mingw32/bitcoin-56e79fe683d3-win64-unsigned.tar.gz 13b537217a13293b93afd7c588b3733955c3188f79a3249b363fb4e885b74b32 guix-build-56e79fe683d3/output/x86_64-w64-mingw32/bitcoin-56e79fe683d3-win64.zip ``` Guix Build (arm64): ```bash e590e67d2b151e7d079f50393c61cacf9e65f5e3fdddc10bffd5dbb59cd9f5e3 guix-build-56e79fe683d3/output/aarch64-linux-gnu/SHA256SUMS.part 9222a733209dc73168722ee0f290831d1723f7ed55abc1e243dd3892ef9733f1 guix-build-56e79fe683d3/output/aarch64-linux-gnu/bitcoin-56e79fe683d3-aarch64-linux-gnu-debug.tar.gz a4ec01411195412e9a483397554d98b2db161a6be8aceee2dac78f0015f2a8e5 guix-build-56e79fe683d3/output/aarch64-linux-gnu/bitcoin-56e79fe683d3-aarch64-linux-gnu.tar.gz a4ff8f2538c1b3a63a3e7c92699c18164b2c4954ebd72c4a854bf7844e6131b6 guix-build-56e79fe683d3/output/arm-linux-gnueabihf/SHA256SUMS.part ac6e4dffcf6f49dac73d3677f3fe5edb08ee1127648bc7ec24b217f48a3a4d21 guix-build-56e79fe683d3/output/arm-linux-gnueabihf/bitcoin-56e79fe683d3-arm-linux-gnueabihf-debug.tar.gz ab26467fce9732596b1713179618eee20c2bd44875dc1c39f9afd68f135e8ce4 guix-build-56e79fe683d3/output/arm-linux-gnueabihf/bitcoin-56e79fe683d3-arm-linux-gnueabihf.tar.gz f44108d818d53ee7e70b44361cb3cc6841b7aecbe4fade1fa289263718620824 guix-build-56e79fe683d3/output/arm64-apple-darwin/SHA256SUMS.part 29adfb2d575be8b5a7092b2aff888af2084ad5783bdc74ae3de4b2dd84da7a26 guix-build-56e79fe683d3/output/arm64-apple-darwin/bitcoin-56e79fe683d3-arm64-apple-darwin-unsigned.dmg 3d7640f604bd27cb1ae47853d70378a7631aa436b5c027eae31fa06d4dd20d4a guix-build-56e79fe683d3/output/arm64-apple-darwin/bitcoin-56e79fe683d3-arm64-apple-darwin-unsigned.tar.gz 3f8d8fc49cf722a7a13640872762d045c14efd610fba908f0acc99a99032b903 guix-build-56e79fe683d3/output/arm64-apple-darwin/bitcoin-56e79fe683d3-arm64-apple-darwin.tar.gz b8df097e10e7acded8556a5681da2b0132c0d509b200792f5584c186b8dcaf24 guix-build-56e79fe683d3/output/dist-archive/bitcoin-56e79fe683d3.tar.gz 1a605c563038107b88e8d859fa045ca3087241340264eb7125e97b3af36874db guix-build-56e79fe683d3/output/powerpc64-linux-gnu/SHA256SUMS.part d10be89a816ee2f47bf778ef4aaa8a7b548403c123a4b4b3aa0470ee7b9472d2 guix-build-56e79fe683d3/output/powerpc64-linux-gnu/bitcoin-56e79fe683d3-powerpc64-linux-gnu-debug.tar.gz 839dd079d677c87a91ccc28f6c423657d98212350d3693ae2facb92082d10c88 guix-build-56e79fe683d3/output/powerpc64-linux-gnu/bitcoin-56e79fe683d3-powerpc64-linux-gnu.tar.gz 200d7a72a7901b6788e70aa1c6fbbe16144e9dac29beb06242a9dcfe1e88acbe guix-build-56e79fe683d3/output/powerpc64le-linux-gnu/SHA256SUMS.part 3d207bb57dc45f234d804a9ee615001aa28ac4b59b530a1e7d16d0a72c2ea514 guix-build-56e79fe683d3/output/powerpc64le-linux-gnu/bitcoin-56e79fe683d3-powerpc64le-linux-gnu-debug.tar.gz 16a789c98409d36bd6b69a5ca38cd8886979f50f845265dc0b005c92286a3e4f guix-build-56e79fe683d3/output/powerpc64le-linux-gnu/bitcoin-56e79fe683d3-powerpc64le-linux-gnu.tar.gz e48f6fae9c2ffeb2bf02ffc073ac69648b2ce4ba87e0ad7a3a5059abf33b8ce2 guix-build-56e79fe683d3/output/riscv64-linux-gnu/SHA256SUMS.part fa8d66af5f202e0c8968114af4dcd6d88099eb91d197dfaaa0144f8ae6b12f4a guix-build-56e79fe683d3/output/riscv64-linux-gnu/bitcoin-56e79fe683d3-riscv64-linux-gnu-debug.tar.gz 86cffa62687f4272c899897ff6c1a86e1a7bdbbb7ab09a085b7b1122d3a1f1b2 guix-build-56e79fe683d3/output/riscv64-linux-gnu/bitcoin-56e79fe683d3-riscv64-linux-gnu.tar.gz e42a7c33da6ea4be5d0826df097bdc7e919c9fd6642e7d59fe3d45a97d9009fa guix-build-56e79fe683d3/output/x86_64-apple-darwin/SHA256SUMS.part 2f9faa88fa0025b9bfade83177b1f50833dc6c3be1ef26c4d5881aa165bcc537 guix-build-56e79fe683d3/output/x86_64-apple-darwin/bitcoin-56e79fe683d3-x86_64-apple-darwin-unsigned.dmg fdfe560968c85500dc886d4e12b851cd27a76de860b8cbf1fb6aac95ccd0582e guix-build-56e79fe683d3/output/x86_64-apple-darwin/bitcoin-56e79fe683d3-x86_64-apple-darwin-unsigned.tar.gz c3c75560befd72a9f13f5626ddf39c63ad25fb9753a4d941e48830b7e06f143e guix-build-56e79fe683d3/output/x86_64-apple-darwin/bitcoin-56e79fe683d3-x86_64-apple-darwin.tar.gz 38100946ec1469fdd31aa8ea9e471b317dc278c298c106289c119016c2c0caa0 guix-build-56e79fe683d3/output/x86_64-linux-gnu/SHA256SUMS.part 67890771aad89dc0d20b8dcdc6b29bd838ace310e8bcbbc0d16d7afc87d3a474 guix-build-56e79fe683d3/output/x86_64-linux-gnu/bitcoin-56e79fe683d3-x86_64-linux-gnu-debug.tar.gz a9772b40c8b9d3c1a0bf9c9db2a60050415bf6a9bd4a19cfd8a038f4a1e712e3 guix-build-56e79fe683d3/output/x86_64-linux-gnu/bitcoin-56e79fe683d3-x86_64-linux-gnu.tar.gz 01da0561f2c3a268c28615b1573dfdecb6346bf3f8eba7a052488191234ac1d6 guix-build-56e79fe683d3/output/x86_64-w64-mingw32/SHA256SUMS.part 71324ac0f6fc8163e41031b2fd2d38d2e15bfd5ef4efe4cb45d7974fad474394 guix-build-56e79fe683d3/output/x86_64-w64-mingw32/bitcoin-56e79fe683d3-win64-debug.zip 219b58281d3c79b2ed7e9085b1e15d7e021fd3899ef07a6ad747058b43d64443 guix-build-56e79fe683d3/output/x86_64-w64-mingw32/bitcoin-56e79fe683d3-win64-setup-unsigned.exe fe838d32587c2f942fcd9f9a0b3735611b686a867e2d2040cf3a8aa6a43d09fe guix-build-56e79fe683d3/output/x86_64-w64-mingw32/bitcoin-56e79fe683d3-win64-unsigned.tar.gz 13b537217a13293b93afd7c588b3733955c3188f79a3249b363fb4e885b74b32 guix-build-56e79fe683d3/output/x86_64-w64-mingw32/bitcoin-56e79fe683d3-win64.zip ``` ACKs for top commit: hebasto: ACK 56e79fe683d36c1944e52326fae3bcc4cb7deec7 Tree-SHA512: 628ab6cda80069ad277107639bef21b44a8417198862e9ec89b45a2c41741d29aeb79aa58c5a90283fb96cf707494ae948ac790abde809bb18c86b14af999200
2022-09-04 19:20:16 +02:00
(xgcc-sans-libc (explicit-cross-configure (cross-gcc target
#:xgcc base-gcc-for-libc
#:xbinutils xbinutils)))
Merge #15277: contrib: Enable building in Guix containers 751549b52a9a4cd27389d807ae67f02bbb39cd7f contrib: guix: Additional clarifications re: substitutes (Carl Dong) cd3e947f50db7cfe05c05b368c25742193729a62 contrib: guix: Various improvements. (Carl Dong) 8dff3e48a9e03299468ed3b342642f01f70da9db contrib: guix: Clarify SOURCE_DATE_EPOCH. (Carl Dong) 3e80ec3ea9691c7c89173de922a113e643fe976b contrib: Add deterministic Guix builds. (Carl Dong) Pull request description: ~~**This post is kept updated as this project progresses. Use this [latest update link](https://github.com/bitcoin/bitcoin/pull/15277#issuecomment-497303718) to see what's new.**~~ Please read the `README.md`. ----- ### Guix Introduction This PR enables building bitcoin in Guix containers. [Guix](https://www.gnu.org/software/guix/manual/en/html_node/Features.html) is a transactional package manager much like Nix, but unlike Nix, it has more of a focus on [bootstrappability](https://www.gnu.org/software/guix/manual/en/html_node/Bootstrapping.html) and [reproducibility](https://www.gnu.org/software/guix/blog/tags/reproducible-builds/) which are attractive for security-sensitive projects like bitcoin. ### Guix Build Walkthrough Please read the `README.md`. [Old instructions no. 4](https://github.com/bitcoin/bitcoin/pull/15277#issuecomment-497303718) [Old instructions no. 3](https://github.com/bitcoin/bitcoin/pull/15277#issuecomment-493827011) [Old instructions no. 2](https://github.com/bitcoin/bitcoin/pull/15277#issuecomment-471658439) <details> <summary>Old instructions no. 1</summary> In this PR, we define a Guix [manifest](https://www.gnu.org/software/guix/manual/en/html_node/Invoking-guix-package.html#profile_002dmanifest) in `contrib/guix/manifest.scm`, which declares what packages we want in our environment. We can then invoke ``` guix environment --manifest=contrib/guix/manifest.scm --container --pure --no-grafts --no-substitutes ``` To have Guix: 1. Build an environment containing the packages we defined in our `contrib/guix/manifest.scm` manifest from the Guix bootstrap binaries (see [bootstrappability](https://www.gnu.org/software/guix/manual/en/html_node/Bootstrapping.html) for more details). 2. Start a container with that environment that has no network access, and no access to the host's filesystem except to the `pwd` that it was started in. 3. Drop you into a shell in that container. > Note: if you don't want to wait hours for Guix to build the entire world from scratch, you can eliminate the `--no-substitutes` option to have Guix download from available binary sources. Note that this convenience doesn't necessarily compromise your security, as you can check that a package was built correctly after the fact using `guix build --check <packagename>` Therefore, we can perform a build of bitcoin much like in Gitian by invoking the following: ``` make -C depends -j"$(nproc)" download && \ cat contrib/guix/build.sh | guix environment --manifest=contrib/guix/manifest.scm --container --pure --no-grafts --no-substitutes ``` We don't include `make -C depends -j"$(nproc)" download` inside `contrib/guix/build.sh` because `contrib/guix/build.sh` is run inside the container, which has no network access (which is a good thing). </details> ### Rationale I believe that this represents a substantial improvement for the "supply chain security" of bitcoin because: 1. We no longer have to rely on Ubuntu for our build environment for our releases ([oh the horror](https://github.com/bitcoin/bitcoin/blob/72bd4ab867e3be0d8410403d9641c08288d343e3/contrib/gitian-descriptors/gitian-linux.yml#L10)), because Guix builds everything about the container, we can perform this on almost any Linux distro/system. 2. It is now much easier to determine what trusted binaries are in our supply chain, and even make a nice visualization! (see [bootstrappability](https://www.gnu.org/software/guix/manual/en/html_node/Bootstrapping.html)). 3. There is active effort among Guix folks to minimize the number of trusted binaries even further. OriansJ's [stage0](https://github.com/oriansj/stage0), and janneke's [Mes](https://www.gnu.org/software/mes/) all aim to achieve [reduced binary boostrap](http://joyofsource.com/reduced-binary-seed-bootstrap.html) for Guix. In fact, I believe if OriansJ gets his way, we will end up some day with only a single trusted binary: hex0 (a ~500 byte self-hosting hex assembler). ### Steps to Completion - [x] Successfully build bitcoin inside the Guix environment - [x] Make `check-symbols` pass - [x] Do the above but without nasty hacks - [x] Solve some of the more innocuous hacks - [ ] Make it cross-compile (HELP WANTED HERE) - [x] Linux - [x] x86_64-linux-gnu - [x] i686-linux-gnu - [x] aarch64-linux-gnu - [x] arm-linux-gnueabihf - [x] riscv64-linux-gnu - [ ] OS X - [ ] x86_64-apple-darwin14 - [ ] Windows - [ ] x86_64-w64-mingw32 - [ ] Maybe make importer for depends syntax - [ ] Document build process for future releases - [ ] Extra: Pin the revision of Guix that we build with with Guix [inferiors](https://www.gnu.org/software/guix/manual/en/html_node/Inferiors.html) ### Help Wanted [Old content no. 3](https://github.com/bitcoin/bitcoin/pull/15277#issuecomment-483318210) [Old content no. 2](https://github.com/bitcoin/bitcoin/pull/15277#issuecomment-471658439) <details> <summary>Old content no. 1</summary> As of now, the command described above to perform a build of bitcoin a lot like Gitian works, but fails at the `check-symbols` stage. This is because a few dynamic libraries are linked in that shouldn't be. Here's what `ldd src/bitcoind` looks like when built in a Guix container: ``` linux-vdso.so.1 (0x00007ffcc2d90000) libdl.so.2 => /gnu/store/h90vnqw0nwd0hhm1l5dgxsdrigddfmq4-glibc-2.28/lib/libdl.so.2 (0x00007fb7eda09000) librt.so.1 => /gnu/store/h90vnqw0nwd0hhm1l5dgxsdrigddfmq4-glibc-2.28/lib/librt.so.1 (0x00007fb7ed9ff000) libstdc++.so.6 => /gnu/store/4sqps8dczv3g7rwbdibfz6rf5jlk7w90-gcc-5.5.0-lib/lib/libstdc++.so.6 (0x00007fb7ed87c000) libpthread.so.0 => /gnu/store/h90vnqw0nwd0hhm1l5dgxsdrigddfmq4-glibc-2.28/lib/libpthread.so.0 (0x00007fb7ed85b000) libm.so.6 => /gnu/store/h90vnqw0nwd0hhm1l5dgxsdrigddfmq4-glibc-2.28/lib/libm.so.6 (0x00007fb7ed6da000) libgcc_s.so.1 => /gnu/store/4sqps8dczv3g7rwbdibfz6rf5jlk7w90-gcc-5.5.0-lib/lib/libgcc_s.so.1 (0x00007fb7ed6bf000) libc.so.6 => /gnu/store/h90vnqw0nwd0hhm1l5dgxsdrigddfmq4-glibc-2.28/lib/libc.so.6 (0x00007fb7ed506000) /gnu/store/h90vnqw0nwd0hhm1l5dgxsdrigddfmq4-glibc-2.28/lib/ld-linux-x86-64.so.2 => /usr/lib64/ld-linux-x86-64.so.2 (0x00007fb7ee3a0000) ``` And here's what it looks in one of our releases: ``` linux-vdso.so.1 (0x00007ffff52cd000) libpthread.so.0 => /usr/lib/libpthread.so.0 (0x00007f87726b4000) librt.so.1 => /usr/lib/librt.so.1 (0x00007f87726aa000) libm.so.6 => /usr/lib/libm.so.6 (0x00007f8772525000) libgcc_s.so.1 => /usr/lib/libgcc_s.so.1 (0x00007f877250b000) libc.so.6 => /usr/lib/libc.so.6 (0x00007f8772347000) /lib64/ld-linux-x86-64.so.2 => /usr/lib64/ld-linux-x86-64.so.2 (0x00007f8773392000) ``` ~~I suspect it is because my script does not apply the gitian-input patches [described in the release process](https://github.com/bitcoin/bitcoin/blob/master/doc/release-process.md#fetch-and-create-inputs-first-time-or-when-dependency-versions-change) but there is no description as to how these patches are applied.~~ It might also be something else entirely. Edit: It is something else. It appears that the gitian inputs are only used by [`gitian-win-signer.yml`](https://github.com/bitcoin/bitcoin/blob/d6e700e40f861ddd6743f4d13f0d6f6bc19093c2/contrib/gitian-descriptors/gitian-win-signer.yml#L14) </details> ### How to Help 1. Install Guix on your distro either [from source](https://www.gnu.org/software/guix/manual/en/html_node/Requirements.html) or perform a [binary installation](https://www.gnu.org/software/guix/manual/en/html_node/Binary-Installation.html#Binary-Installation) 2. Try out my branch and the command described above! ACKs for top commit: MarcoFalke: Thanks for the replies. ACK 751549b52a9a4cd27389d807ae67f02bbb39cd7f laanwj: ACK 751549b52a9a4cd27389d807ae67f02bbb39cd7f Tree-SHA512: 50e6ab58c6bda9a67125b6271daf7eff0ca57d0efa8941ed3cd951e5bf78b31552fc5e537b1e1bcf2d3cc918c63adf19d685aa117a0f851024dc67e697890a8d
2019-07-12 19:23:30 +02:00
;; 2. Build cross-compiled kernel headers with XGCC-SANS-LIBC, derived
;; from BASE-KERNEL-HEADERS
(xkernel (cross-kernel-headers target
base-kernel-headers
xgcc-sans-libc
xbinutils))
;; 3. Build a cross-compiled libc with XGCC-SANS-LIBC and XKERNEL,
;; derived from BASE-LIBC
Merge bitcoin/bitcoin#25861: guix: use --build={arch}-guix-linux-gnu in cross toolchain 56e79fe683d36c1944e52326fae3bcc4cb7deec7 guix: use --build={arch}-guix-linux-gnu in cross toolchain (fanquake) Pull request description: Technically we are always cross-compiling when Guix building, so make that explicit. `{arch}-guix-linux-gnu` is not a triplet that should be used in any other capacity, but here it serves the purpose of ensuring, that by setting `--build` to something other than `--host/--target`, we are always cross-compiling (in the eyes of autoconf etc) when building our cross toolchains. It looks like `x86_64-linux-gnu` on `x86_64-linux-gnu` currently works because of the triplet canonicalisation, i.e `x86_64-linux-gnu` becomes `x86_64-pc-linux-gnu`, and GCCs configure thinking it's cross-compiling, whereas the same canonicalisation doesn't happen for `aarch64-linux-gnu` so we don't get a cross-compile when building on aarch64. Fixes: #22458. Guix Build (x86_64): ```bash e590e67d2b151e7d079f50393c61cacf9e65f5e3fdddc10bffd5dbb59cd9f5e3 guix-build-56e79fe683d3/output/aarch64-linux-gnu/SHA256SUMS.part 9222a733209dc73168722ee0f290831d1723f7ed55abc1e243dd3892ef9733f1 guix-build-56e79fe683d3/output/aarch64-linux-gnu/bitcoin-56e79fe683d3-aarch64-linux-gnu-debug.tar.gz a4ec01411195412e9a483397554d98b2db161a6be8aceee2dac78f0015f2a8e5 guix-build-56e79fe683d3/output/aarch64-linux-gnu/bitcoin-56e79fe683d3-aarch64-linux-gnu.tar.gz a4ff8f2538c1b3a63a3e7c92699c18164b2c4954ebd72c4a854bf7844e6131b6 guix-build-56e79fe683d3/output/arm-linux-gnueabihf/SHA256SUMS.part ac6e4dffcf6f49dac73d3677f3fe5edb08ee1127648bc7ec24b217f48a3a4d21 guix-build-56e79fe683d3/output/arm-linux-gnueabihf/bitcoin-56e79fe683d3-arm-linux-gnueabihf-debug.tar.gz ab26467fce9732596b1713179618eee20c2bd44875dc1c39f9afd68f135e8ce4 guix-build-56e79fe683d3/output/arm-linux-gnueabihf/bitcoin-56e79fe683d3-arm-linux-gnueabihf.tar.gz f44108d818d53ee7e70b44361cb3cc6841b7aecbe4fade1fa289263718620824 guix-build-56e79fe683d3/output/arm64-apple-darwin/SHA256SUMS.part 29adfb2d575be8b5a7092b2aff888af2084ad5783bdc74ae3de4b2dd84da7a26 guix-build-56e79fe683d3/output/arm64-apple-darwin/bitcoin-56e79fe683d3-arm64-apple-darwin-unsigned.dmg 3d7640f604bd27cb1ae47853d70378a7631aa436b5c027eae31fa06d4dd20d4a guix-build-56e79fe683d3/output/arm64-apple-darwin/bitcoin-56e79fe683d3-arm64-apple-darwin-unsigned.tar.gz 3f8d8fc49cf722a7a13640872762d045c14efd610fba908f0acc99a99032b903 guix-build-56e79fe683d3/output/arm64-apple-darwin/bitcoin-56e79fe683d3-arm64-apple-darwin.tar.gz b8df097e10e7acded8556a5681da2b0132c0d509b200792f5584c186b8dcaf24 guix-build-56e79fe683d3/output/dist-archive/bitcoin-56e79fe683d3.tar.gz 1a605c563038107b88e8d859fa045ca3087241340264eb7125e97b3af36874db guix-build-56e79fe683d3/output/powerpc64-linux-gnu/SHA256SUMS.part d10be89a816ee2f47bf778ef4aaa8a7b548403c123a4b4b3aa0470ee7b9472d2 guix-build-56e79fe683d3/output/powerpc64-linux-gnu/bitcoin-56e79fe683d3-powerpc64-linux-gnu-debug.tar.gz 839dd079d677c87a91ccc28f6c423657d98212350d3693ae2facb92082d10c88 guix-build-56e79fe683d3/output/powerpc64-linux-gnu/bitcoin-56e79fe683d3-powerpc64-linux-gnu.tar.gz 200d7a72a7901b6788e70aa1c6fbbe16144e9dac29beb06242a9dcfe1e88acbe guix-build-56e79fe683d3/output/powerpc64le-linux-gnu/SHA256SUMS.part 3d207bb57dc45f234d804a9ee615001aa28ac4b59b530a1e7d16d0a72c2ea514 guix-build-56e79fe683d3/output/powerpc64le-linux-gnu/bitcoin-56e79fe683d3-powerpc64le-linux-gnu-debug.tar.gz 16a789c98409d36bd6b69a5ca38cd8886979f50f845265dc0b005c92286a3e4f guix-build-56e79fe683d3/output/powerpc64le-linux-gnu/bitcoin-56e79fe683d3-powerpc64le-linux-gnu.tar.gz e48f6fae9c2ffeb2bf02ffc073ac69648b2ce4ba87e0ad7a3a5059abf33b8ce2 guix-build-56e79fe683d3/output/riscv64-linux-gnu/SHA256SUMS.part fa8d66af5f202e0c8968114af4dcd6d88099eb91d197dfaaa0144f8ae6b12f4a guix-build-56e79fe683d3/output/riscv64-linux-gnu/bitcoin-56e79fe683d3-riscv64-linux-gnu-debug.tar.gz 86cffa62687f4272c899897ff6c1a86e1a7bdbbb7ab09a085b7b1122d3a1f1b2 guix-build-56e79fe683d3/output/riscv64-linux-gnu/bitcoin-56e79fe683d3-riscv64-linux-gnu.tar.gz e42a7c33da6ea4be5d0826df097bdc7e919c9fd6642e7d59fe3d45a97d9009fa guix-build-56e79fe683d3/output/x86_64-apple-darwin/SHA256SUMS.part 2f9faa88fa0025b9bfade83177b1f50833dc6c3be1ef26c4d5881aa165bcc537 guix-build-56e79fe683d3/output/x86_64-apple-darwin/bitcoin-56e79fe683d3-x86_64-apple-darwin-unsigned.dmg fdfe560968c85500dc886d4e12b851cd27a76de860b8cbf1fb6aac95ccd0582e guix-build-56e79fe683d3/output/x86_64-apple-darwin/bitcoin-56e79fe683d3-x86_64-apple-darwin-unsigned.tar.gz c3c75560befd72a9f13f5626ddf39c63ad25fb9753a4d941e48830b7e06f143e guix-build-56e79fe683d3/output/x86_64-apple-darwin/bitcoin-56e79fe683d3-x86_64-apple-darwin.tar.gz 38100946ec1469fdd31aa8ea9e471b317dc278c298c106289c119016c2c0caa0 guix-build-56e79fe683d3/output/x86_64-linux-gnu/SHA256SUMS.part 67890771aad89dc0d20b8dcdc6b29bd838ace310e8bcbbc0d16d7afc87d3a474 guix-build-56e79fe683d3/output/x86_64-linux-gnu/bitcoin-56e79fe683d3-x86_64-linux-gnu-debug.tar.gz a9772b40c8b9d3c1a0bf9c9db2a60050415bf6a9bd4a19cfd8a038f4a1e712e3 guix-build-56e79fe683d3/output/x86_64-linux-gnu/bitcoin-56e79fe683d3-x86_64-linux-gnu.tar.gz 01da0561f2c3a268c28615b1573dfdecb6346bf3f8eba7a052488191234ac1d6 guix-build-56e79fe683d3/output/x86_64-w64-mingw32/SHA256SUMS.part 71324ac0f6fc8163e41031b2fd2d38d2e15bfd5ef4efe4cb45d7974fad474394 guix-build-56e79fe683d3/output/x86_64-w64-mingw32/bitcoin-56e79fe683d3-win64-debug.zip 219b58281d3c79b2ed7e9085b1e15d7e021fd3899ef07a6ad747058b43d64443 guix-build-56e79fe683d3/output/x86_64-w64-mingw32/bitcoin-56e79fe683d3-win64-setup-unsigned.exe fe838d32587c2f942fcd9f9a0b3735611b686a867e2d2040cf3a8aa6a43d09fe guix-build-56e79fe683d3/output/x86_64-w64-mingw32/bitcoin-56e79fe683d3-win64-unsigned.tar.gz 13b537217a13293b93afd7c588b3733955c3188f79a3249b363fb4e885b74b32 guix-build-56e79fe683d3/output/x86_64-w64-mingw32/bitcoin-56e79fe683d3-win64.zip ``` Guix Build (arm64): ```bash e590e67d2b151e7d079f50393c61cacf9e65f5e3fdddc10bffd5dbb59cd9f5e3 guix-build-56e79fe683d3/output/aarch64-linux-gnu/SHA256SUMS.part 9222a733209dc73168722ee0f290831d1723f7ed55abc1e243dd3892ef9733f1 guix-build-56e79fe683d3/output/aarch64-linux-gnu/bitcoin-56e79fe683d3-aarch64-linux-gnu-debug.tar.gz a4ec01411195412e9a483397554d98b2db161a6be8aceee2dac78f0015f2a8e5 guix-build-56e79fe683d3/output/aarch64-linux-gnu/bitcoin-56e79fe683d3-aarch64-linux-gnu.tar.gz a4ff8f2538c1b3a63a3e7c92699c18164b2c4954ebd72c4a854bf7844e6131b6 guix-build-56e79fe683d3/output/arm-linux-gnueabihf/SHA256SUMS.part ac6e4dffcf6f49dac73d3677f3fe5edb08ee1127648bc7ec24b217f48a3a4d21 guix-build-56e79fe683d3/output/arm-linux-gnueabihf/bitcoin-56e79fe683d3-arm-linux-gnueabihf-debug.tar.gz ab26467fce9732596b1713179618eee20c2bd44875dc1c39f9afd68f135e8ce4 guix-build-56e79fe683d3/output/arm-linux-gnueabihf/bitcoin-56e79fe683d3-arm-linux-gnueabihf.tar.gz f44108d818d53ee7e70b44361cb3cc6841b7aecbe4fade1fa289263718620824 guix-build-56e79fe683d3/output/arm64-apple-darwin/SHA256SUMS.part 29adfb2d575be8b5a7092b2aff888af2084ad5783bdc74ae3de4b2dd84da7a26 guix-build-56e79fe683d3/output/arm64-apple-darwin/bitcoin-56e79fe683d3-arm64-apple-darwin-unsigned.dmg 3d7640f604bd27cb1ae47853d70378a7631aa436b5c027eae31fa06d4dd20d4a guix-build-56e79fe683d3/output/arm64-apple-darwin/bitcoin-56e79fe683d3-arm64-apple-darwin-unsigned.tar.gz 3f8d8fc49cf722a7a13640872762d045c14efd610fba908f0acc99a99032b903 guix-build-56e79fe683d3/output/arm64-apple-darwin/bitcoin-56e79fe683d3-arm64-apple-darwin.tar.gz b8df097e10e7acded8556a5681da2b0132c0d509b200792f5584c186b8dcaf24 guix-build-56e79fe683d3/output/dist-archive/bitcoin-56e79fe683d3.tar.gz 1a605c563038107b88e8d859fa045ca3087241340264eb7125e97b3af36874db guix-build-56e79fe683d3/output/powerpc64-linux-gnu/SHA256SUMS.part d10be89a816ee2f47bf778ef4aaa8a7b548403c123a4b4b3aa0470ee7b9472d2 guix-build-56e79fe683d3/output/powerpc64-linux-gnu/bitcoin-56e79fe683d3-powerpc64-linux-gnu-debug.tar.gz 839dd079d677c87a91ccc28f6c423657d98212350d3693ae2facb92082d10c88 guix-build-56e79fe683d3/output/powerpc64-linux-gnu/bitcoin-56e79fe683d3-powerpc64-linux-gnu.tar.gz 200d7a72a7901b6788e70aa1c6fbbe16144e9dac29beb06242a9dcfe1e88acbe guix-build-56e79fe683d3/output/powerpc64le-linux-gnu/SHA256SUMS.part 3d207bb57dc45f234d804a9ee615001aa28ac4b59b530a1e7d16d0a72c2ea514 guix-build-56e79fe683d3/output/powerpc64le-linux-gnu/bitcoin-56e79fe683d3-powerpc64le-linux-gnu-debug.tar.gz 16a789c98409d36bd6b69a5ca38cd8886979f50f845265dc0b005c92286a3e4f guix-build-56e79fe683d3/output/powerpc64le-linux-gnu/bitcoin-56e79fe683d3-powerpc64le-linux-gnu.tar.gz e48f6fae9c2ffeb2bf02ffc073ac69648b2ce4ba87e0ad7a3a5059abf33b8ce2 guix-build-56e79fe683d3/output/riscv64-linux-gnu/SHA256SUMS.part fa8d66af5f202e0c8968114af4dcd6d88099eb91d197dfaaa0144f8ae6b12f4a guix-build-56e79fe683d3/output/riscv64-linux-gnu/bitcoin-56e79fe683d3-riscv64-linux-gnu-debug.tar.gz 86cffa62687f4272c899897ff6c1a86e1a7bdbbb7ab09a085b7b1122d3a1f1b2 guix-build-56e79fe683d3/output/riscv64-linux-gnu/bitcoin-56e79fe683d3-riscv64-linux-gnu.tar.gz e42a7c33da6ea4be5d0826df097bdc7e919c9fd6642e7d59fe3d45a97d9009fa guix-build-56e79fe683d3/output/x86_64-apple-darwin/SHA256SUMS.part 2f9faa88fa0025b9bfade83177b1f50833dc6c3be1ef26c4d5881aa165bcc537 guix-build-56e79fe683d3/output/x86_64-apple-darwin/bitcoin-56e79fe683d3-x86_64-apple-darwin-unsigned.dmg fdfe560968c85500dc886d4e12b851cd27a76de860b8cbf1fb6aac95ccd0582e guix-build-56e79fe683d3/output/x86_64-apple-darwin/bitcoin-56e79fe683d3-x86_64-apple-darwin-unsigned.tar.gz c3c75560befd72a9f13f5626ddf39c63ad25fb9753a4d941e48830b7e06f143e guix-build-56e79fe683d3/output/x86_64-apple-darwin/bitcoin-56e79fe683d3-x86_64-apple-darwin.tar.gz 38100946ec1469fdd31aa8ea9e471b317dc278c298c106289c119016c2c0caa0 guix-build-56e79fe683d3/output/x86_64-linux-gnu/SHA256SUMS.part 67890771aad89dc0d20b8dcdc6b29bd838ace310e8bcbbc0d16d7afc87d3a474 guix-build-56e79fe683d3/output/x86_64-linux-gnu/bitcoin-56e79fe683d3-x86_64-linux-gnu-debug.tar.gz a9772b40c8b9d3c1a0bf9c9db2a60050415bf6a9bd4a19cfd8a038f4a1e712e3 guix-build-56e79fe683d3/output/x86_64-linux-gnu/bitcoin-56e79fe683d3-x86_64-linux-gnu.tar.gz 01da0561f2c3a268c28615b1573dfdecb6346bf3f8eba7a052488191234ac1d6 guix-build-56e79fe683d3/output/x86_64-w64-mingw32/SHA256SUMS.part 71324ac0f6fc8163e41031b2fd2d38d2e15bfd5ef4efe4cb45d7974fad474394 guix-build-56e79fe683d3/output/x86_64-w64-mingw32/bitcoin-56e79fe683d3-win64-debug.zip 219b58281d3c79b2ed7e9085b1e15d7e021fd3899ef07a6ad747058b43d64443 guix-build-56e79fe683d3/output/x86_64-w64-mingw32/bitcoin-56e79fe683d3-win64-setup-unsigned.exe fe838d32587c2f942fcd9f9a0b3735611b686a867e2d2040cf3a8aa6a43d09fe guix-build-56e79fe683d3/output/x86_64-w64-mingw32/bitcoin-56e79fe683d3-win64-unsigned.tar.gz 13b537217a13293b93afd7c588b3733955c3188f79a3249b363fb4e885b74b32 guix-build-56e79fe683d3/output/x86_64-w64-mingw32/bitcoin-56e79fe683d3-win64.zip ``` ACKs for top commit: hebasto: ACK 56e79fe683d36c1944e52326fae3bcc4cb7deec7 Tree-SHA512: 628ab6cda80069ad277107639bef21b44a8417198862e9ec89b45a2c41741d29aeb79aa58c5a90283fb96cf707494ae948ac790abde809bb18c86b14af999200
2022-09-04 19:20:16 +02:00
(xlibc (explicit-cross-configure (cross-libc target
base-libc
xgcc-sans-libc
xbinutils
xkernel)))
Merge #15277: contrib: Enable building in Guix containers 751549b52a9a4cd27389d807ae67f02bbb39cd7f contrib: guix: Additional clarifications re: substitutes (Carl Dong) cd3e947f50db7cfe05c05b368c25742193729a62 contrib: guix: Various improvements. (Carl Dong) 8dff3e48a9e03299468ed3b342642f01f70da9db contrib: guix: Clarify SOURCE_DATE_EPOCH. (Carl Dong) 3e80ec3ea9691c7c89173de922a113e643fe976b contrib: Add deterministic Guix builds. (Carl Dong) Pull request description: ~~**This post is kept updated as this project progresses. Use this [latest update link](https://github.com/bitcoin/bitcoin/pull/15277#issuecomment-497303718) to see what's new.**~~ Please read the `README.md`. ----- ### Guix Introduction This PR enables building bitcoin in Guix containers. [Guix](https://www.gnu.org/software/guix/manual/en/html_node/Features.html) is a transactional package manager much like Nix, but unlike Nix, it has more of a focus on [bootstrappability](https://www.gnu.org/software/guix/manual/en/html_node/Bootstrapping.html) and [reproducibility](https://www.gnu.org/software/guix/blog/tags/reproducible-builds/) which are attractive for security-sensitive projects like bitcoin. ### Guix Build Walkthrough Please read the `README.md`. [Old instructions no. 4](https://github.com/bitcoin/bitcoin/pull/15277#issuecomment-497303718) [Old instructions no. 3](https://github.com/bitcoin/bitcoin/pull/15277#issuecomment-493827011) [Old instructions no. 2](https://github.com/bitcoin/bitcoin/pull/15277#issuecomment-471658439) <details> <summary>Old instructions no. 1</summary> In this PR, we define a Guix [manifest](https://www.gnu.org/software/guix/manual/en/html_node/Invoking-guix-package.html#profile_002dmanifest) in `contrib/guix/manifest.scm`, which declares what packages we want in our environment. We can then invoke ``` guix environment --manifest=contrib/guix/manifest.scm --container --pure --no-grafts --no-substitutes ``` To have Guix: 1. Build an environment containing the packages we defined in our `contrib/guix/manifest.scm` manifest from the Guix bootstrap binaries (see [bootstrappability](https://www.gnu.org/software/guix/manual/en/html_node/Bootstrapping.html) for more details). 2. Start a container with that environment that has no network access, and no access to the host's filesystem except to the `pwd` that it was started in. 3. Drop you into a shell in that container. > Note: if you don't want to wait hours for Guix to build the entire world from scratch, you can eliminate the `--no-substitutes` option to have Guix download from available binary sources. Note that this convenience doesn't necessarily compromise your security, as you can check that a package was built correctly after the fact using `guix build --check <packagename>` Therefore, we can perform a build of bitcoin much like in Gitian by invoking the following: ``` make -C depends -j"$(nproc)" download && \ cat contrib/guix/build.sh | guix environment --manifest=contrib/guix/manifest.scm --container --pure --no-grafts --no-substitutes ``` We don't include `make -C depends -j"$(nproc)" download` inside `contrib/guix/build.sh` because `contrib/guix/build.sh` is run inside the container, which has no network access (which is a good thing). </details> ### Rationale I believe that this represents a substantial improvement for the "supply chain security" of bitcoin because: 1. We no longer have to rely on Ubuntu for our build environment for our releases ([oh the horror](https://github.com/bitcoin/bitcoin/blob/72bd4ab867e3be0d8410403d9641c08288d343e3/contrib/gitian-descriptors/gitian-linux.yml#L10)), because Guix builds everything about the container, we can perform this on almost any Linux distro/system. 2. It is now much easier to determine what trusted binaries are in our supply chain, and even make a nice visualization! (see [bootstrappability](https://www.gnu.org/software/guix/manual/en/html_node/Bootstrapping.html)). 3. There is active effort among Guix folks to minimize the number of trusted binaries even further. OriansJ's [stage0](https://github.com/oriansj/stage0), and janneke's [Mes](https://www.gnu.org/software/mes/) all aim to achieve [reduced binary boostrap](http://joyofsource.com/reduced-binary-seed-bootstrap.html) for Guix. In fact, I believe if OriansJ gets his way, we will end up some day with only a single trusted binary: hex0 (a ~500 byte self-hosting hex assembler). ### Steps to Completion - [x] Successfully build bitcoin inside the Guix environment - [x] Make `check-symbols` pass - [x] Do the above but without nasty hacks - [x] Solve some of the more innocuous hacks - [ ] Make it cross-compile (HELP WANTED HERE) - [x] Linux - [x] x86_64-linux-gnu - [x] i686-linux-gnu - [x] aarch64-linux-gnu - [x] arm-linux-gnueabihf - [x] riscv64-linux-gnu - [ ] OS X - [ ] x86_64-apple-darwin14 - [ ] Windows - [ ] x86_64-w64-mingw32 - [ ] Maybe make importer for depends syntax - [ ] Document build process for future releases - [ ] Extra: Pin the revision of Guix that we build with with Guix [inferiors](https://www.gnu.org/software/guix/manual/en/html_node/Inferiors.html) ### Help Wanted [Old content no. 3](https://github.com/bitcoin/bitcoin/pull/15277#issuecomment-483318210) [Old content no. 2](https://github.com/bitcoin/bitcoin/pull/15277#issuecomment-471658439) <details> <summary>Old content no. 1</summary> As of now, the command described above to perform a build of bitcoin a lot like Gitian works, but fails at the `check-symbols` stage. This is because a few dynamic libraries are linked in that shouldn't be. Here's what `ldd src/bitcoind` looks like when built in a Guix container: ``` linux-vdso.so.1 (0x00007ffcc2d90000) libdl.so.2 => /gnu/store/h90vnqw0nwd0hhm1l5dgxsdrigddfmq4-glibc-2.28/lib/libdl.so.2 (0x00007fb7eda09000) librt.so.1 => /gnu/store/h90vnqw0nwd0hhm1l5dgxsdrigddfmq4-glibc-2.28/lib/librt.so.1 (0x00007fb7ed9ff000) libstdc++.so.6 => /gnu/store/4sqps8dczv3g7rwbdibfz6rf5jlk7w90-gcc-5.5.0-lib/lib/libstdc++.so.6 (0x00007fb7ed87c000) libpthread.so.0 => /gnu/store/h90vnqw0nwd0hhm1l5dgxsdrigddfmq4-glibc-2.28/lib/libpthread.so.0 (0x00007fb7ed85b000) libm.so.6 => /gnu/store/h90vnqw0nwd0hhm1l5dgxsdrigddfmq4-glibc-2.28/lib/libm.so.6 (0x00007fb7ed6da000) libgcc_s.so.1 => /gnu/store/4sqps8dczv3g7rwbdibfz6rf5jlk7w90-gcc-5.5.0-lib/lib/libgcc_s.so.1 (0x00007fb7ed6bf000) libc.so.6 => /gnu/store/h90vnqw0nwd0hhm1l5dgxsdrigddfmq4-glibc-2.28/lib/libc.so.6 (0x00007fb7ed506000) /gnu/store/h90vnqw0nwd0hhm1l5dgxsdrigddfmq4-glibc-2.28/lib/ld-linux-x86-64.so.2 => /usr/lib64/ld-linux-x86-64.so.2 (0x00007fb7ee3a0000) ``` And here's what it looks in one of our releases: ``` linux-vdso.so.1 (0x00007ffff52cd000) libpthread.so.0 => /usr/lib/libpthread.so.0 (0x00007f87726b4000) librt.so.1 => /usr/lib/librt.so.1 (0x00007f87726aa000) libm.so.6 => /usr/lib/libm.so.6 (0x00007f8772525000) libgcc_s.so.1 => /usr/lib/libgcc_s.so.1 (0x00007f877250b000) libc.so.6 => /usr/lib/libc.so.6 (0x00007f8772347000) /lib64/ld-linux-x86-64.so.2 => /usr/lib64/ld-linux-x86-64.so.2 (0x00007f8773392000) ``` ~~I suspect it is because my script does not apply the gitian-input patches [described in the release process](https://github.com/bitcoin/bitcoin/blob/master/doc/release-process.md#fetch-and-create-inputs-first-time-or-when-dependency-versions-change) but there is no description as to how these patches are applied.~~ It might also be something else entirely. Edit: It is something else. It appears that the gitian inputs are only used by [`gitian-win-signer.yml`](https://github.com/bitcoin/bitcoin/blob/d6e700e40f861ddd6743f4d13f0d6f6bc19093c2/contrib/gitian-descriptors/gitian-win-signer.yml#L14) </details> ### How to Help 1. Install Guix on your distro either [from source](https://www.gnu.org/software/guix/manual/en/html_node/Requirements.html) or perform a [binary installation](https://www.gnu.org/software/guix/manual/en/html_node/Binary-Installation.html#Binary-Installation) 2. Try out my branch and the command described above! ACKs for top commit: MarcoFalke: Thanks for the replies. ACK 751549b52a9a4cd27389d807ae67f02bbb39cd7f laanwj: ACK 751549b52a9a4cd27389d807ae67f02bbb39cd7f Tree-SHA512: 50e6ab58c6bda9a67125b6271daf7eff0ca57d0efa8941ed3cd951e5bf78b31552fc5e537b1e1bcf2d3cc918c63adf19d685aa117a0f851024dc67e697890a8d
2019-07-12 19:23:30 +02:00
;; 4. Build a cross-compiling gcc targeting XLIBC, derived from
;; BASE-GCC
Merge bitcoin/bitcoin#25861: guix: use --build={arch}-guix-linux-gnu in cross toolchain 56e79fe683d36c1944e52326fae3bcc4cb7deec7 guix: use --build={arch}-guix-linux-gnu in cross toolchain (fanquake) Pull request description: Technically we are always cross-compiling when Guix building, so make that explicit. `{arch}-guix-linux-gnu` is not a triplet that should be used in any other capacity, but here it serves the purpose of ensuring, that by setting `--build` to something other than `--host/--target`, we are always cross-compiling (in the eyes of autoconf etc) when building our cross toolchains. It looks like `x86_64-linux-gnu` on `x86_64-linux-gnu` currently works because of the triplet canonicalisation, i.e `x86_64-linux-gnu` becomes `x86_64-pc-linux-gnu`, and GCCs configure thinking it's cross-compiling, whereas the same canonicalisation doesn't happen for `aarch64-linux-gnu` so we don't get a cross-compile when building on aarch64. Fixes: #22458. Guix Build (x86_64): ```bash e590e67d2b151e7d079f50393c61cacf9e65f5e3fdddc10bffd5dbb59cd9f5e3 guix-build-56e79fe683d3/output/aarch64-linux-gnu/SHA256SUMS.part 9222a733209dc73168722ee0f290831d1723f7ed55abc1e243dd3892ef9733f1 guix-build-56e79fe683d3/output/aarch64-linux-gnu/bitcoin-56e79fe683d3-aarch64-linux-gnu-debug.tar.gz a4ec01411195412e9a483397554d98b2db161a6be8aceee2dac78f0015f2a8e5 guix-build-56e79fe683d3/output/aarch64-linux-gnu/bitcoin-56e79fe683d3-aarch64-linux-gnu.tar.gz a4ff8f2538c1b3a63a3e7c92699c18164b2c4954ebd72c4a854bf7844e6131b6 guix-build-56e79fe683d3/output/arm-linux-gnueabihf/SHA256SUMS.part ac6e4dffcf6f49dac73d3677f3fe5edb08ee1127648bc7ec24b217f48a3a4d21 guix-build-56e79fe683d3/output/arm-linux-gnueabihf/bitcoin-56e79fe683d3-arm-linux-gnueabihf-debug.tar.gz ab26467fce9732596b1713179618eee20c2bd44875dc1c39f9afd68f135e8ce4 guix-build-56e79fe683d3/output/arm-linux-gnueabihf/bitcoin-56e79fe683d3-arm-linux-gnueabihf.tar.gz f44108d818d53ee7e70b44361cb3cc6841b7aecbe4fade1fa289263718620824 guix-build-56e79fe683d3/output/arm64-apple-darwin/SHA256SUMS.part 29adfb2d575be8b5a7092b2aff888af2084ad5783bdc74ae3de4b2dd84da7a26 guix-build-56e79fe683d3/output/arm64-apple-darwin/bitcoin-56e79fe683d3-arm64-apple-darwin-unsigned.dmg 3d7640f604bd27cb1ae47853d70378a7631aa436b5c027eae31fa06d4dd20d4a guix-build-56e79fe683d3/output/arm64-apple-darwin/bitcoin-56e79fe683d3-arm64-apple-darwin-unsigned.tar.gz 3f8d8fc49cf722a7a13640872762d045c14efd610fba908f0acc99a99032b903 guix-build-56e79fe683d3/output/arm64-apple-darwin/bitcoin-56e79fe683d3-arm64-apple-darwin.tar.gz b8df097e10e7acded8556a5681da2b0132c0d509b200792f5584c186b8dcaf24 guix-build-56e79fe683d3/output/dist-archive/bitcoin-56e79fe683d3.tar.gz 1a605c563038107b88e8d859fa045ca3087241340264eb7125e97b3af36874db guix-build-56e79fe683d3/output/powerpc64-linux-gnu/SHA256SUMS.part d10be89a816ee2f47bf778ef4aaa8a7b548403c123a4b4b3aa0470ee7b9472d2 guix-build-56e79fe683d3/output/powerpc64-linux-gnu/bitcoin-56e79fe683d3-powerpc64-linux-gnu-debug.tar.gz 839dd079d677c87a91ccc28f6c423657d98212350d3693ae2facb92082d10c88 guix-build-56e79fe683d3/output/powerpc64-linux-gnu/bitcoin-56e79fe683d3-powerpc64-linux-gnu.tar.gz 200d7a72a7901b6788e70aa1c6fbbe16144e9dac29beb06242a9dcfe1e88acbe guix-build-56e79fe683d3/output/powerpc64le-linux-gnu/SHA256SUMS.part 3d207bb57dc45f234d804a9ee615001aa28ac4b59b530a1e7d16d0a72c2ea514 guix-build-56e79fe683d3/output/powerpc64le-linux-gnu/bitcoin-56e79fe683d3-powerpc64le-linux-gnu-debug.tar.gz 16a789c98409d36bd6b69a5ca38cd8886979f50f845265dc0b005c92286a3e4f guix-build-56e79fe683d3/output/powerpc64le-linux-gnu/bitcoin-56e79fe683d3-powerpc64le-linux-gnu.tar.gz e48f6fae9c2ffeb2bf02ffc073ac69648b2ce4ba87e0ad7a3a5059abf33b8ce2 guix-build-56e79fe683d3/output/riscv64-linux-gnu/SHA256SUMS.part fa8d66af5f202e0c8968114af4dcd6d88099eb91d197dfaaa0144f8ae6b12f4a guix-build-56e79fe683d3/output/riscv64-linux-gnu/bitcoin-56e79fe683d3-riscv64-linux-gnu-debug.tar.gz 86cffa62687f4272c899897ff6c1a86e1a7bdbbb7ab09a085b7b1122d3a1f1b2 guix-build-56e79fe683d3/output/riscv64-linux-gnu/bitcoin-56e79fe683d3-riscv64-linux-gnu.tar.gz e42a7c33da6ea4be5d0826df097bdc7e919c9fd6642e7d59fe3d45a97d9009fa guix-build-56e79fe683d3/output/x86_64-apple-darwin/SHA256SUMS.part 2f9faa88fa0025b9bfade83177b1f50833dc6c3be1ef26c4d5881aa165bcc537 guix-build-56e79fe683d3/output/x86_64-apple-darwin/bitcoin-56e79fe683d3-x86_64-apple-darwin-unsigned.dmg fdfe560968c85500dc886d4e12b851cd27a76de860b8cbf1fb6aac95ccd0582e guix-build-56e79fe683d3/output/x86_64-apple-darwin/bitcoin-56e79fe683d3-x86_64-apple-darwin-unsigned.tar.gz c3c75560befd72a9f13f5626ddf39c63ad25fb9753a4d941e48830b7e06f143e guix-build-56e79fe683d3/output/x86_64-apple-darwin/bitcoin-56e79fe683d3-x86_64-apple-darwin.tar.gz 38100946ec1469fdd31aa8ea9e471b317dc278c298c106289c119016c2c0caa0 guix-build-56e79fe683d3/output/x86_64-linux-gnu/SHA256SUMS.part 67890771aad89dc0d20b8dcdc6b29bd838ace310e8bcbbc0d16d7afc87d3a474 guix-build-56e79fe683d3/output/x86_64-linux-gnu/bitcoin-56e79fe683d3-x86_64-linux-gnu-debug.tar.gz a9772b40c8b9d3c1a0bf9c9db2a60050415bf6a9bd4a19cfd8a038f4a1e712e3 guix-build-56e79fe683d3/output/x86_64-linux-gnu/bitcoin-56e79fe683d3-x86_64-linux-gnu.tar.gz 01da0561f2c3a268c28615b1573dfdecb6346bf3f8eba7a052488191234ac1d6 guix-build-56e79fe683d3/output/x86_64-w64-mingw32/SHA256SUMS.part 71324ac0f6fc8163e41031b2fd2d38d2e15bfd5ef4efe4cb45d7974fad474394 guix-build-56e79fe683d3/output/x86_64-w64-mingw32/bitcoin-56e79fe683d3-win64-debug.zip 219b58281d3c79b2ed7e9085b1e15d7e021fd3899ef07a6ad747058b43d64443 guix-build-56e79fe683d3/output/x86_64-w64-mingw32/bitcoin-56e79fe683d3-win64-setup-unsigned.exe fe838d32587c2f942fcd9f9a0b3735611b686a867e2d2040cf3a8aa6a43d09fe guix-build-56e79fe683d3/output/x86_64-w64-mingw32/bitcoin-56e79fe683d3-win64-unsigned.tar.gz 13b537217a13293b93afd7c588b3733955c3188f79a3249b363fb4e885b74b32 guix-build-56e79fe683d3/output/x86_64-w64-mingw32/bitcoin-56e79fe683d3-win64.zip ``` Guix Build (arm64): ```bash e590e67d2b151e7d079f50393c61cacf9e65f5e3fdddc10bffd5dbb59cd9f5e3 guix-build-56e79fe683d3/output/aarch64-linux-gnu/SHA256SUMS.part 9222a733209dc73168722ee0f290831d1723f7ed55abc1e243dd3892ef9733f1 guix-build-56e79fe683d3/output/aarch64-linux-gnu/bitcoin-56e79fe683d3-aarch64-linux-gnu-debug.tar.gz a4ec01411195412e9a483397554d98b2db161a6be8aceee2dac78f0015f2a8e5 guix-build-56e79fe683d3/output/aarch64-linux-gnu/bitcoin-56e79fe683d3-aarch64-linux-gnu.tar.gz a4ff8f2538c1b3a63a3e7c92699c18164b2c4954ebd72c4a854bf7844e6131b6 guix-build-56e79fe683d3/output/arm-linux-gnueabihf/SHA256SUMS.part ac6e4dffcf6f49dac73d3677f3fe5edb08ee1127648bc7ec24b217f48a3a4d21 guix-build-56e79fe683d3/output/arm-linux-gnueabihf/bitcoin-56e79fe683d3-arm-linux-gnueabihf-debug.tar.gz ab26467fce9732596b1713179618eee20c2bd44875dc1c39f9afd68f135e8ce4 guix-build-56e79fe683d3/output/arm-linux-gnueabihf/bitcoin-56e79fe683d3-arm-linux-gnueabihf.tar.gz f44108d818d53ee7e70b44361cb3cc6841b7aecbe4fade1fa289263718620824 guix-build-56e79fe683d3/output/arm64-apple-darwin/SHA256SUMS.part 29adfb2d575be8b5a7092b2aff888af2084ad5783bdc74ae3de4b2dd84da7a26 guix-build-56e79fe683d3/output/arm64-apple-darwin/bitcoin-56e79fe683d3-arm64-apple-darwin-unsigned.dmg 3d7640f604bd27cb1ae47853d70378a7631aa436b5c027eae31fa06d4dd20d4a guix-build-56e79fe683d3/output/arm64-apple-darwin/bitcoin-56e79fe683d3-arm64-apple-darwin-unsigned.tar.gz 3f8d8fc49cf722a7a13640872762d045c14efd610fba908f0acc99a99032b903 guix-build-56e79fe683d3/output/arm64-apple-darwin/bitcoin-56e79fe683d3-arm64-apple-darwin.tar.gz b8df097e10e7acded8556a5681da2b0132c0d509b200792f5584c186b8dcaf24 guix-build-56e79fe683d3/output/dist-archive/bitcoin-56e79fe683d3.tar.gz 1a605c563038107b88e8d859fa045ca3087241340264eb7125e97b3af36874db guix-build-56e79fe683d3/output/powerpc64-linux-gnu/SHA256SUMS.part d10be89a816ee2f47bf778ef4aaa8a7b548403c123a4b4b3aa0470ee7b9472d2 guix-build-56e79fe683d3/output/powerpc64-linux-gnu/bitcoin-56e79fe683d3-powerpc64-linux-gnu-debug.tar.gz 839dd079d677c87a91ccc28f6c423657d98212350d3693ae2facb92082d10c88 guix-build-56e79fe683d3/output/powerpc64-linux-gnu/bitcoin-56e79fe683d3-powerpc64-linux-gnu.tar.gz 200d7a72a7901b6788e70aa1c6fbbe16144e9dac29beb06242a9dcfe1e88acbe guix-build-56e79fe683d3/output/powerpc64le-linux-gnu/SHA256SUMS.part 3d207bb57dc45f234d804a9ee615001aa28ac4b59b530a1e7d16d0a72c2ea514 guix-build-56e79fe683d3/output/powerpc64le-linux-gnu/bitcoin-56e79fe683d3-powerpc64le-linux-gnu-debug.tar.gz 16a789c98409d36bd6b69a5ca38cd8886979f50f845265dc0b005c92286a3e4f guix-build-56e79fe683d3/output/powerpc64le-linux-gnu/bitcoin-56e79fe683d3-powerpc64le-linux-gnu.tar.gz e48f6fae9c2ffeb2bf02ffc073ac69648b2ce4ba87e0ad7a3a5059abf33b8ce2 guix-build-56e79fe683d3/output/riscv64-linux-gnu/SHA256SUMS.part fa8d66af5f202e0c8968114af4dcd6d88099eb91d197dfaaa0144f8ae6b12f4a guix-build-56e79fe683d3/output/riscv64-linux-gnu/bitcoin-56e79fe683d3-riscv64-linux-gnu-debug.tar.gz 86cffa62687f4272c899897ff6c1a86e1a7bdbbb7ab09a085b7b1122d3a1f1b2 guix-build-56e79fe683d3/output/riscv64-linux-gnu/bitcoin-56e79fe683d3-riscv64-linux-gnu.tar.gz e42a7c33da6ea4be5d0826df097bdc7e919c9fd6642e7d59fe3d45a97d9009fa guix-build-56e79fe683d3/output/x86_64-apple-darwin/SHA256SUMS.part 2f9faa88fa0025b9bfade83177b1f50833dc6c3be1ef26c4d5881aa165bcc537 guix-build-56e79fe683d3/output/x86_64-apple-darwin/bitcoin-56e79fe683d3-x86_64-apple-darwin-unsigned.dmg fdfe560968c85500dc886d4e12b851cd27a76de860b8cbf1fb6aac95ccd0582e guix-build-56e79fe683d3/output/x86_64-apple-darwin/bitcoin-56e79fe683d3-x86_64-apple-darwin-unsigned.tar.gz c3c75560befd72a9f13f5626ddf39c63ad25fb9753a4d941e48830b7e06f143e guix-build-56e79fe683d3/output/x86_64-apple-darwin/bitcoin-56e79fe683d3-x86_64-apple-darwin.tar.gz 38100946ec1469fdd31aa8ea9e471b317dc278c298c106289c119016c2c0caa0 guix-build-56e79fe683d3/output/x86_64-linux-gnu/SHA256SUMS.part 67890771aad89dc0d20b8dcdc6b29bd838ace310e8bcbbc0d16d7afc87d3a474 guix-build-56e79fe683d3/output/x86_64-linux-gnu/bitcoin-56e79fe683d3-x86_64-linux-gnu-debug.tar.gz a9772b40c8b9d3c1a0bf9c9db2a60050415bf6a9bd4a19cfd8a038f4a1e712e3 guix-build-56e79fe683d3/output/x86_64-linux-gnu/bitcoin-56e79fe683d3-x86_64-linux-gnu.tar.gz 01da0561f2c3a268c28615b1573dfdecb6346bf3f8eba7a052488191234ac1d6 guix-build-56e79fe683d3/output/x86_64-w64-mingw32/SHA256SUMS.part 71324ac0f6fc8163e41031b2fd2d38d2e15bfd5ef4efe4cb45d7974fad474394 guix-build-56e79fe683d3/output/x86_64-w64-mingw32/bitcoin-56e79fe683d3-win64-debug.zip 219b58281d3c79b2ed7e9085b1e15d7e021fd3899ef07a6ad747058b43d64443 guix-build-56e79fe683d3/output/x86_64-w64-mingw32/bitcoin-56e79fe683d3-win64-setup-unsigned.exe fe838d32587c2f942fcd9f9a0b3735611b686a867e2d2040cf3a8aa6a43d09fe guix-build-56e79fe683d3/output/x86_64-w64-mingw32/bitcoin-56e79fe683d3-win64-unsigned.tar.gz 13b537217a13293b93afd7c588b3733955c3188f79a3249b363fb4e885b74b32 guix-build-56e79fe683d3/output/x86_64-w64-mingw32/bitcoin-56e79fe683d3-win64.zip ``` ACKs for top commit: hebasto: ACK 56e79fe683d36c1944e52326fae3bcc4cb7deec7 Tree-SHA512: 628ab6cda80069ad277107639bef21b44a8417198862e9ec89b45a2c41741d29aeb79aa58c5a90283fb96cf707494ae948ac790abde809bb18c86b14af999200
2022-09-04 19:20:16 +02:00
(xgcc (explicit-cross-configure (cross-gcc target
#:xgcc base-gcc
#:xbinutils xbinutils
#:libc xlibc))))
Merge #15277: contrib: Enable building in Guix containers 751549b52a9a4cd27389d807ae67f02bbb39cd7f contrib: guix: Additional clarifications re: substitutes (Carl Dong) cd3e947f50db7cfe05c05b368c25742193729a62 contrib: guix: Various improvements. (Carl Dong) 8dff3e48a9e03299468ed3b342642f01f70da9db contrib: guix: Clarify SOURCE_DATE_EPOCH. (Carl Dong) 3e80ec3ea9691c7c89173de922a113e643fe976b contrib: Add deterministic Guix builds. (Carl Dong) Pull request description: ~~**This post is kept updated as this project progresses. Use this [latest update link](https://github.com/bitcoin/bitcoin/pull/15277#issuecomment-497303718) to see what's new.**~~ Please read the `README.md`. ----- ### Guix Introduction This PR enables building bitcoin in Guix containers. [Guix](https://www.gnu.org/software/guix/manual/en/html_node/Features.html) is a transactional package manager much like Nix, but unlike Nix, it has more of a focus on [bootstrappability](https://www.gnu.org/software/guix/manual/en/html_node/Bootstrapping.html) and [reproducibility](https://www.gnu.org/software/guix/blog/tags/reproducible-builds/) which are attractive for security-sensitive projects like bitcoin. ### Guix Build Walkthrough Please read the `README.md`. [Old instructions no. 4](https://github.com/bitcoin/bitcoin/pull/15277#issuecomment-497303718) [Old instructions no. 3](https://github.com/bitcoin/bitcoin/pull/15277#issuecomment-493827011) [Old instructions no. 2](https://github.com/bitcoin/bitcoin/pull/15277#issuecomment-471658439) <details> <summary>Old instructions no. 1</summary> In this PR, we define a Guix [manifest](https://www.gnu.org/software/guix/manual/en/html_node/Invoking-guix-package.html#profile_002dmanifest) in `contrib/guix/manifest.scm`, which declares what packages we want in our environment. We can then invoke ``` guix environment --manifest=contrib/guix/manifest.scm --container --pure --no-grafts --no-substitutes ``` To have Guix: 1. Build an environment containing the packages we defined in our `contrib/guix/manifest.scm` manifest from the Guix bootstrap binaries (see [bootstrappability](https://www.gnu.org/software/guix/manual/en/html_node/Bootstrapping.html) for more details). 2. Start a container with that environment that has no network access, and no access to the host's filesystem except to the `pwd` that it was started in. 3. Drop you into a shell in that container. > Note: if you don't want to wait hours for Guix to build the entire world from scratch, you can eliminate the `--no-substitutes` option to have Guix download from available binary sources. Note that this convenience doesn't necessarily compromise your security, as you can check that a package was built correctly after the fact using `guix build --check <packagename>` Therefore, we can perform a build of bitcoin much like in Gitian by invoking the following: ``` make -C depends -j"$(nproc)" download && \ cat contrib/guix/build.sh | guix environment --manifest=contrib/guix/manifest.scm --container --pure --no-grafts --no-substitutes ``` We don't include `make -C depends -j"$(nproc)" download` inside `contrib/guix/build.sh` because `contrib/guix/build.sh` is run inside the container, which has no network access (which is a good thing). </details> ### Rationale I believe that this represents a substantial improvement for the "supply chain security" of bitcoin because: 1. We no longer have to rely on Ubuntu for our build environment for our releases ([oh the horror](https://github.com/bitcoin/bitcoin/blob/72bd4ab867e3be0d8410403d9641c08288d343e3/contrib/gitian-descriptors/gitian-linux.yml#L10)), because Guix builds everything about the container, we can perform this on almost any Linux distro/system. 2. It is now much easier to determine what trusted binaries are in our supply chain, and even make a nice visualization! (see [bootstrappability](https://www.gnu.org/software/guix/manual/en/html_node/Bootstrapping.html)). 3. There is active effort among Guix folks to minimize the number of trusted binaries even further. OriansJ's [stage0](https://github.com/oriansj/stage0), and janneke's [Mes](https://www.gnu.org/software/mes/) all aim to achieve [reduced binary boostrap](http://joyofsource.com/reduced-binary-seed-bootstrap.html) for Guix. In fact, I believe if OriansJ gets his way, we will end up some day with only a single trusted binary: hex0 (a ~500 byte self-hosting hex assembler). ### Steps to Completion - [x] Successfully build bitcoin inside the Guix environment - [x] Make `check-symbols` pass - [x] Do the above but without nasty hacks - [x] Solve some of the more innocuous hacks - [ ] Make it cross-compile (HELP WANTED HERE) - [x] Linux - [x] x86_64-linux-gnu - [x] i686-linux-gnu - [x] aarch64-linux-gnu - [x] arm-linux-gnueabihf - [x] riscv64-linux-gnu - [ ] OS X - [ ] x86_64-apple-darwin14 - [ ] Windows - [ ] x86_64-w64-mingw32 - [ ] Maybe make importer for depends syntax - [ ] Document build process for future releases - [ ] Extra: Pin the revision of Guix that we build with with Guix [inferiors](https://www.gnu.org/software/guix/manual/en/html_node/Inferiors.html) ### Help Wanted [Old content no. 3](https://github.com/bitcoin/bitcoin/pull/15277#issuecomment-483318210) [Old content no. 2](https://github.com/bitcoin/bitcoin/pull/15277#issuecomment-471658439) <details> <summary>Old content no. 1</summary> As of now, the command described above to perform a build of bitcoin a lot like Gitian works, but fails at the `check-symbols` stage. This is because a few dynamic libraries are linked in that shouldn't be. Here's what `ldd src/bitcoind` looks like when built in a Guix container: ``` linux-vdso.so.1 (0x00007ffcc2d90000) libdl.so.2 => /gnu/store/h90vnqw0nwd0hhm1l5dgxsdrigddfmq4-glibc-2.28/lib/libdl.so.2 (0x00007fb7eda09000) librt.so.1 => /gnu/store/h90vnqw0nwd0hhm1l5dgxsdrigddfmq4-glibc-2.28/lib/librt.so.1 (0x00007fb7ed9ff000) libstdc++.so.6 => /gnu/store/4sqps8dczv3g7rwbdibfz6rf5jlk7w90-gcc-5.5.0-lib/lib/libstdc++.so.6 (0x00007fb7ed87c000) libpthread.so.0 => /gnu/store/h90vnqw0nwd0hhm1l5dgxsdrigddfmq4-glibc-2.28/lib/libpthread.so.0 (0x00007fb7ed85b000) libm.so.6 => /gnu/store/h90vnqw0nwd0hhm1l5dgxsdrigddfmq4-glibc-2.28/lib/libm.so.6 (0x00007fb7ed6da000) libgcc_s.so.1 => /gnu/store/4sqps8dczv3g7rwbdibfz6rf5jlk7w90-gcc-5.5.0-lib/lib/libgcc_s.so.1 (0x00007fb7ed6bf000) libc.so.6 => /gnu/store/h90vnqw0nwd0hhm1l5dgxsdrigddfmq4-glibc-2.28/lib/libc.so.6 (0x00007fb7ed506000) /gnu/store/h90vnqw0nwd0hhm1l5dgxsdrigddfmq4-glibc-2.28/lib/ld-linux-x86-64.so.2 => /usr/lib64/ld-linux-x86-64.so.2 (0x00007fb7ee3a0000) ``` And here's what it looks in one of our releases: ``` linux-vdso.so.1 (0x00007ffff52cd000) libpthread.so.0 => /usr/lib/libpthread.so.0 (0x00007f87726b4000) librt.so.1 => /usr/lib/librt.so.1 (0x00007f87726aa000) libm.so.6 => /usr/lib/libm.so.6 (0x00007f8772525000) libgcc_s.so.1 => /usr/lib/libgcc_s.so.1 (0x00007f877250b000) libc.so.6 => /usr/lib/libc.so.6 (0x00007f8772347000) /lib64/ld-linux-x86-64.so.2 => /usr/lib64/ld-linux-x86-64.so.2 (0x00007f8773392000) ``` ~~I suspect it is because my script does not apply the gitian-input patches [described in the release process](https://github.com/bitcoin/bitcoin/blob/master/doc/release-process.md#fetch-and-create-inputs-first-time-or-when-dependency-versions-change) but there is no description as to how these patches are applied.~~ It might also be something else entirely. Edit: It is something else. It appears that the gitian inputs are only used by [`gitian-win-signer.yml`](https://github.com/bitcoin/bitcoin/blob/d6e700e40f861ddd6743f4d13f0d6f6bc19093c2/contrib/gitian-descriptors/gitian-win-signer.yml#L14) </details> ### How to Help 1. Install Guix on your distro either [from source](https://www.gnu.org/software/guix/manual/en/html_node/Requirements.html) or perform a [binary installation](https://www.gnu.org/software/guix/manual/en/html_node/Binary-Installation.html#Binary-Installation) 2. Try out my branch and the command described above! ACKs for top commit: MarcoFalke: Thanks for the replies. ACK 751549b52a9a4cd27389d807ae67f02bbb39cd7f laanwj: ACK 751549b52a9a4cd27389d807ae67f02bbb39cd7f Tree-SHA512: 50e6ab58c6bda9a67125b6271daf7eff0ca57d0efa8941ed3cd951e5bf78b31552fc5e537b1e1bcf2d3cc918c63adf19d685aa117a0f851024dc67e697890a8d
2019-07-12 19:23:30 +02:00
;; Define a meta-package that propagates the resulting XBINUTILS, XLIBC, and
;; XGCC
(package
(name (string-append target "-toolchain"))
(version (package-version xgcc))
(source #f)
(build-system trivial-build-system)
(arguments '(#:builder (begin (mkdir %output) #t)))
(propagated-inputs
`(("binutils" ,xbinutils)
("libc" ,xlibc)
("libc:static" ,xlibc "static")
("gcc" ,xgcc)
("gcc-lib" ,xgcc "lib")))
Merge #15277: contrib: Enable building in Guix containers 751549b52a9a4cd27389d807ae67f02bbb39cd7f contrib: guix: Additional clarifications re: substitutes (Carl Dong) cd3e947f50db7cfe05c05b368c25742193729a62 contrib: guix: Various improvements. (Carl Dong) 8dff3e48a9e03299468ed3b342642f01f70da9db contrib: guix: Clarify SOURCE_DATE_EPOCH. (Carl Dong) 3e80ec3ea9691c7c89173de922a113e643fe976b contrib: Add deterministic Guix builds. (Carl Dong) Pull request description: ~~**This post is kept updated as this project progresses. Use this [latest update link](https://github.com/bitcoin/bitcoin/pull/15277#issuecomment-497303718) to see what's new.**~~ Please read the `README.md`. ----- ### Guix Introduction This PR enables building bitcoin in Guix containers. [Guix](https://www.gnu.org/software/guix/manual/en/html_node/Features.html) is a transactional package manager much like Nix, but unlike Nix, it has more of a focus on [bootstrappability](https://www.gnu.org/software/guix/manual/en/html_node/Bootstrapping.html) and [reproducibility](https://www.gnu.org/software/guix/blog/tags/reproducible-builds/) which are attractive for security-sensitive projects like bitcoin. ### Guix Build Walkthrough Please read the `README.md`. [Old instructions no. 4](https://github.com/bitcoin/bitcoin/pull/15277#issuecomment-497303718) [Old instructions no. 3](https://github.com/bitcoin/bitcoin/pull/15277#issuecomment-493827011) [Old instructions no. 2](https://github.com/bitcoin/bitcoin/pull/15277#issuecomment-471658439) <details> <summary>Old instructions no. 1</summary> In this PR, we define a Guix [manifest](https://www.gnu.org/software/guix/manual/en/html_node/Invoking-guix-package.html#profile_002dmanifest) in `contrib/guix/manifest.scm`, which declares what packages we want in our environment. We can then invoke ``` guix environment --manifest=contrib/guix/manifest.scm --container --pure --no-grafts --no-substitutes ``` To have Guix: 1. Build an environment containing the packages we defined in our `contrib/guix/manifest.scm` manifest from the Guix bootstrap binaries (see [bootstrappability](https://www.gnu.org/software/guix/manual/en/html_node/Bootstrapping.html) for more details). 2. Start a container with that environment that has no network access, and no access to the host's filesystem except to the `pwd` that it was started in. 3. Drop you into a shell in that container. > Note: if you don't want to wait hours for Guix to build the entire world from scratch, you can eliminate the `--no-substitutes` option to have Guix download from available binary sources. Note that this convenience doesn't necessarily compromise your security, as you can check that a package was built correctly after the fact using `guix build --check <packagename>` Therefore, we can perform a build of bitcoin much like in Gitian by invoking the following: ``` make -C depends -j"$(nproc)" download && \ cat contrib/guix/build.sh | guix environment --manifest=contrib/guix/manifest.scm --container --pure --no-grafts --no-substitutes ``` We don't include `make -C depends -j"$(nproc)" download` inside `contrib/guix/build.sh` because `contrib/guix/build.sh` is run inside the container, which has no network access (which is a good thing). </details> ### Rationale I believe that this represents a substantial improvement for the "supply chain security" of bitcoin because: 1. We no longer have to rely on Ubuntu for our build environment for our releases ([oh the horror](https://github.com/bitcoin/bitcoin/blob/72bd4ab867e3be0d8410403d9641c08288d343e3/contrib/gitian-descriptors/gitian-linux.yml#L10)), because Guix builds everything about the container, we can perform this on almost any Linux distro/system. 2. It is now much easier to determine what trusted binaries are in our supply chain, and even make a nice visualization! (see [bootstrappability](https://www.gnu.org/software/guix/manual/en/html_node/Bootstrapping.html)). 3. There is active effort among Guix folks to minimize the number of trusted binaries even further. OriansJ's [stage0](https://github.com/oriansj/stage0), and janneke's [Mes](https://www.gnu.org/software/mes/) all aim to achieve [reduced binary boostrap](http://joyofsource.com/reduced-binary-seed-bootstrap.html) for Guix. In fact, I believe if OriansJ gets his way, we will end up some day with only a single trusted binary: hex0 (a ~500 byte self-hosting hex assembler). ### Steps to Completion - [x] Successfully build bitcoin inside the Guix environment - [x] Make `check-symbols` pass - [x] Do the above but without nasty hacks - [x] Solve some of the more innocuous hacks - [ ] Make it cross-compile (HELP WANTED HERE) - [x] Linux - [x] x86_64-linux-gnu - [x] i686-linux-gnu - [x] aarch64-linux-gnu - [x] arm-linux-gnueabihf - [x] riscv64-linux-gnu - [ ] OS X - [ ] x86_64-apple-darwin14 - [ ] Windows - [ ] x86_64-w64-mingw32 - [ ] Maybe make importer for depends syntax - [ ] Document build process for future releases - [ ] Extra: Pin the revision of Guix that we build with with Guix [inferiors](https://www.gnu.org/software/guix/manual/en/html_node/Inferiors.html) ### Help Wanted [Old content no. 3](https://github.com/bitcoin/bitcoin/pull/15277#issuecomment-483318210) [Old content no. 2](https://github.com/bitcoin/bitcoin/pull/15277#issuecomment-471658439) <details> <summary>Old content no. 1</summary> As of now, the command described above to perform a build of bitcoin a lot like Gitian works, but fails at the `check-symbols` stage. This is because a few dynamic libraries are linked in that shouldn't be. Here's what `ldd src/bitcoind` looks like when built in a Guix container: ``` linux-vdso.so.1 (0x00007ffcc2d90000) libdl.so.2 => /gnu/store/h90vnqw0nwd0hhm1l5dgxsdrigddfmq4-glibc-2.28/lib/libdl.so.2 (0x00007fb7eda09000) librt.so.1 => /gnu/store/h90vnqw0nwd0hhm1l5dgxsdrigddfmq4-glibc-2.28/lib/librt.so.1 (0x00007fb7ed9ff000) libstdc++.so.6 => /gnu/store/4sqps8dczv3g7rwbdibfz6rf5jlk7w90-gcc-5.5.0-lib/lib/libstdc++.so.6 (0x00007fb7ed87c000) libpthread.so.0 => /gnu/store/h90vnqw0nwd0hhm1l5dgxsdrigddfmq4-glibc-2.28/lib/libpthread.so.0 (0x00007fb7ed85b000) libm.so.6 => /gnu/store/h90vnqw0nwd0hhm1l5dgxsdrigddfmq4-glibc-2.28/lib/libm.so.6 (0x00007fb7ed6da000) libgcc_s.so.1 => /gnu/store/4sqps8dczv3g7rwbdibfz6rf5jlk7w90-gcc-5.5.0-lib/lib/libgcc_s.so.1 (0x00007fb7ed6bf000) libc.so.6 => /gnu/store/h90vnqw0nwd0hhm1l5dgxsdrigddfmq4-glibc-2.28/lib/libc.so.6 (0x00007fb7ed506000) /gnu/store/h90vnqw0nwd0hhm1l5dgxsdrigddfmq4-glibc-2.28/lib/ld-linux-x86-64.so.2 => /usr/lib64/ld-linux-x86-64.so.2 (0x00007fb7ee3a0000) ``` And here's what it looks in one of our releases: ``` linux-vdso.so.1 (0x00007ffff52cd000) libpthread.so.0 => /usr/lib/libpthread.so.0 (0x00007f87726b4000) librt.so.1 => /usr/lib/librt.so.1 (0x00007f87726aa000) libm.so.6 => /usr/lib/libm.so.6 (0x00007f8772525000) libgcc_s.so.1 => /usr/lib/libgcc_s.so.1 (0x00007f877250b000) libc.so.6 => /usr/lib/libc.so.6 (0x00007f8772347000) /lib64/ld-linux-x86-64.so.2 => /usr/lib64/ld-linux-x86-64.so.2 (0x00007f8773392000) ``` ~~I suspect it is because my script does not apply the gitian-input patches [described in the release process](https://github.com/bitcoin/bitcoin/blob/master/doc/release-process.md#fetch-and-create-inputs-first-time-or-when-dependency-versions-change) but there is no description as to how these patches are applied.~~ It might also be something else entirely. Edit: It is something else. It appears that the gitian inputs are only used by [`gitian-win-signer.yml`](https://github.com/bitcoin/bitcoin/blob/d6e700e40f861ddd6743f4d13f0d6f6bc19093c2/contrib/gitian-descriptors/gitian-win-signer.yml#L14) </details> ### How to Help 1. Install Guix on your distro either [from source](https://www.gnu.org/software/guix/manual/en/html_node/Requirements.html) or perform a [binary installation](https://www.gnu.org/software/guix/manual/en/html_node/Binary-Installation.html#Binary-Installation) 2. Try out my branch and the command described above! ACKs for top commit: MarcoFalke: Thanks for the replies. ACK 751549b52a9a4cd27389d807ae67f02bbb39cd7f laanwj: ACK 751549b52a9a4cd27389d807ae67f02bbb39cd7f Tree-SHA512: 50e6ab58c6bda9a67125b6271daf7eff0ca57d0efa8941ed3cd951e5bf78b31552fc5e537b1e1bcf2d3cc918c63adf19d685aa117a0f851024dc67e697890a8d
2019-07-12 19:23:30 +02:00
(synopsis (string-append "Complete GCC tool chain for " target))
(description (string-append "This package provides a complete GCC tool
chain for " target " development."))
(home-page (package-home-page xgcc))
(license (package-license xgcc)))))
Merge bitcoin/bitcoin#21239: guix: Add codesignature attachment support for osx+win ee883201cf134952284632e9e9ae72bf1c8c792f guix: repro: Sort find output in libtool for gcc-8 (Carl Dong) ee0a67c32a8861eab650bf8894af06807578eba0 codesigning: Use SHA256 as digest for osslsigncode (Windows) (Carl Dong) 38eb91eb0616ed6dbe34c23e11588d130fef07f8 guix: Add codesigning functionality (Carl Dong) bac2690e6f683fcedb883fe1d32f3c33c628a141 guix: Package codesigning tools (Carl Dong) 0a2176d47767972e4cd5ed302c1dbeedece1708b guix: Reindent existing manifest.scm (Carl Dong) c090a3e9238ba2df07875b4708e908d8dca4ed9b Makefile.am: use APP_DIST_DIR instead of hard-coding dist (Carl Dong) Pull request description: This is the last PR before we reach feature-parity with the Gitian process! Note: I tried using the `Makefile` inside the distsrc to make the dmg instead of manually listing out the commands, but `make` seems to want to re-make a lot of other files which broke the dmg. The workflow looks something like this: 1. `env [ FOO=bar... ] ./contrib/guix/guix-build` (add additional env vars as necessary) 2. Codesigners only: 1. Copy `guix-build-<short-id>/output/x86_64-apple-darwin18/bitcoin-<short-id>-osx-unsigned.tar.gz` and `guix-build-<short-id>/output/x86_64-w64-mingw32/bitcoin-<short-id>-win-unsigned.tar.gz` to signing computer 2. Codesign with `./detached-sig-create.sh` inside the tarball 3. Upload contents of `signature-{osx,win}.tar.gz` to https://github.com/bitcoin-core/bitcoin-detached-sigs (as a new tag) 3. Checkout new tag for `bitcoin-core/bitcoin-detached-sigs` with the detached signatures 4. `env [ FOO=bar... ] DETACHED_SIGS_REPO=<path/to/bitcoin-detached-sigs> ./contrib/guix/guix-codesign` (modify env vars as necessary) 5. Make sure `guix.sigs` is cloned and updated 6. `env GUIX_SIGS_REPO=<path/to/guix.sigs> SIGNER=0x96AB007F1A7ED999=dongcarl ./contrib/guix/guix-attest` (modify env vars as necessary) 7. Commit your new signatures and SHA256SUMS in `guix.sigs` 8. Optionally, after there are multiple signatures in `guix.sigs`: `env GUIX_SIGS_REPO=<path/to/guix.sigs> ./contrib/guix/guix-verify` ACKs for top commit: laanwj: Tested ACK ee883201cf134952284632e9e9ae72bf1c8c792f achow101: ACK ee883201cf134952284632e9e9ae72bf1c8c792f Tree-SHA512: e812a07a5f19f900600c70cb9c717769ef544a6c0c12760b5558b76b6b37df863257f3dbf38b0757e6e06e334470267e94c9f2bdbc27409d6837b1a0bfc6acbc
2021-05-24 15:31:05 +02:00
(define base-gcc
(package-with-extra-patches gcc-8
(search-our-patches "gcc-8-sort-libtool-find-output.patch")))
;; Building glibc with stack smashing protector first landed in glibc 2.25, use
;; this function to disable for older glibcs
;;
;; From glibc 2.25 changelog:
;;
;; * Most of glibc can now be built with the stack smashing protector enabled.
;; It is recommended to build glibc with --enable-stack-protector=strong.
;; Implemented by Nick Alcock (Oracle).
(define (make-glibc-without-ssp xglibc)
(package-with-extra-configure-variable
(package-with-extra-configure-variable
xglibc "libc_cv_ssp" "no")
"libc_cv_ssp_strong" "no"))
Merge #15277: contrib: Enable building in Guix containers 751549b52a9a4cd27389d807ae67f02bbb39cd7f contrib: guix: Additional clarifications re: substitutes (Carl Dong) cd3e947f50db7cfe05c05b368c25742193729a62 contrib: guix: Various improvements. (Carl Dong) 8dff3e48a9e03299468ed3b342642f01f70da9db contrib: guix: Clarify SOURCE_DATE_EPOCH. (Carl Dong) 3e80ec3ea9691c7c89173de922a113e643fe976b contrib: Add deterministic Guix builds. (Carl Dong) Pull request description: ~~**This post is kept updated as this project progresses. Use this [latest update link](https://github.com/bitcoin/bitcoin/pull/15277#issuecomment-497303718) to see what's new.**~~ Please read the `README.md`. ----- ### Guix Introduction This PR enables building bitcoin in Guix containers. [Guix](https://www.gnu.org/software/guix/manual/en/html_node/Features.html) is a transactional package manager much like Nix, but unlike Nix, it has more of a focus on [bootstrappability](https://www.gnu.org/software/guix/manual/en/html_node/Bootstrapping.html) and [reproducibility](https://www.gnu.org/software/guix/blog/tags/reproducible-builds/) which are attractive for security-sensitive projects like bitcoin. ### Guix Build Walkthrough Please read the `README.md`. [Old instructions no. 4](https://github.com/bitcoin/bitcoin/pull/15277#issuecomment-497303718) [Old instructions no. 3](https://github.com/bitcoin/bitcoin/pull/15277#issuecomment-493827011) [Old instructions no. 2](https://github.com/bitcoin/bitcoin/pull/15277#issuecomment-471658439) <details> <summary>Old instructions no. 1</summary> In this PR, we define a Guix [manifest](https://www.gnu.org/software/guix/manual/en/html_node/Invoking-guix-package.html#profile_002dmanifest) in `contrib/guix/manifest.scm`, which declares what packages we want in our environment. We can then invoke ``` guix environment --manifest=contrib/guix/manifest.scm --container --pure --no-grafts --no-substitutes ``` To have Guix: 1. Build an environment containing the packages we defined in our `contrib/guix/manifest.scm` manifest from the Guix bootstrap binaries (see [bootstrappability](https://www.gnu.org/software/guix/manual/en/html_node/Bootstrapping.html) for more details). 2. Start a container with that environment that has no network access, and no access to the host's filesystem except to the `pwd` that it was started in. 3. Drop you into a shell in that container. > Note: if you don't want to wait hours for Guix to build the entire world from scratch, you can eliminate the `--no-substitutes` option to have Guix download from available binary sources. Note that this convenience doesn't necessarily compromise your security, as you can check that a package was built correctly after the fact using `guix build --check <packagename>` Therefore, we can perform a build of bitcoin much like in Gitian by invoking the following: ``` make -C depends -j"$(nproc)" download && \ cat contrib/guix/build.sh | guix environment --manifest=contrib/guix/manifest.scm --container --pure --no-grafts --no-substitutes ``` We don't include `make -C depends -j"$(nproc)" download` inside `contrib/guix/build.sh` because `contrib/guix/build.sh` is run inside the container, which has no network access (which is a good thing). </details> ### Rationale I believe that this represents a substantial improvement for the "supply chain security" of bitcoin because: 1. We no longer have to rely on Ubuntu for our build environment for our releases ([oh the horror](https://github.com/bitcoin/bitcoin/blob/72bd4ab867e3be0d8410403d9641c08288d343e3/contrib/gitian-descriptors/gitian-linux.yml#L10)), because Guix builds everything about the container, we can perform this on almost any Linux distro/system. 2. It is now much easier to determine what trusted binaries are in our supply chain, and even make a nice visualization! (see [bootstrappability](https://www.gnu.org/software/guix/manual/en/html_node/Bootstrapping.html)). 3. There is active effort among Guix folks to minimize the number of trusted binaries even further. OriansJ's [stage0](https://github.com/oriansj/stage0), and janneke's [Mes](https://www.gnu.org/software/mes/) all aim to achieve [reduced binary boostrap](http://joyofsource.com/reduced-binary-seed-bootstrap.html) for Guix. In fact, I believe if OriansJ gets his way, we will end up some day with only a single trusted binary: hex0 (a ~500 byte self-hosting hex assembler). ### Steps to Completion - [x] Successfully build bitcoin inside the Guix environment - [x] Make `check-symbols` pass - [x] Do the above but without nasty hacks - [x] Solve some of the more innocuous hacks - [ ] Make it cross-compile (HELP WANTED HERE) - [x] Linux - [x] x86_64-linux-gnu - [x] i686-linux-gnu - [x] aarch64-linux-gnu - [x] arm-linux-gnueabihf - [x] riscv64-linux-gnu - [ ] OS X - [ ] x86_64-apple-darwin14 - [ ] Windows - [ ] x86_64-w64-mingw32 - [ ] Maybe make importer for depends syntax - [ ] Document build process for future releases - [ ] Extra: Pin the revision of Guix that we build with with Guix [inferiors](https://www.gnu.org/software/guix/manual/en/html_node/Inferiors.html) ### Help Wanted [Old content no. 3](https://github.com/bitcoin/bitcoin/pull/15277#issuecomment-483318210) [Old content no. 2](https://github.com/bitcoin/bitcoin/pull/15277#issuecomment-471658439) <details> <summary>Old content no. 1</summary> As of now, the command described above to perform a build of bitcoin a lot like Gitian works, but fails at the `check-symbols` stage. This is because a few dynamic libraries are linked in that shouldn't be. Here's what `ldd src/bitcoind` looks like when built in a Guix container: ``` linux-vdso.so.1 (0x00007ffcc2d90000) libdl.so.2 => /gnu/store/h90vnqw0nwd0hhm1l5dgxsdrigddfmq4-glibc-2.28/lib/libdl.so.2 (0x00007fb7eda09000) librt.so.1 => /gnu/store/h90vnqw0nwd0hhm1l5dgxsdrigddfmq4-glibc-2.28/lib/librt.so.1 (0x00007fb7ed9ff000) libstdc++.so.6 => /gnu/store/4sqps8dczv3g7rwbdibfz6rf5jlk7w90-gcc-5.5.0-lib/lib/libstdc++.so.6 (0x00007fb7ed87c000) libpthread.so.0 => /gnu/store/h90vnqw0nwd0hhm1l5dgxsdrigddfmq4-glibc-2.28/lib/libpthread.so.0 (0x00007fb7ed85b000) libm.so.6 => /gnu/store/h90vnqw0nwd0hhm1l5dgxsdrigddfmq4-glibc-2.28/lib/libm.so.6 (0x00007fb7ed6da000) libgcc_s.so.1 => /gnu/store/4sqps8dczv3g7rwbdibfz6rf5jlk7w90-gcc-5.5.0-lib/lib/libgcc_s.so.1 (0x00007fb7ed6bf000) libc.so.6 => /gnu/store/h90vnqw0nwd0hhm1l5dgxsdrigddfmq4-glibc-2.28/lib/libc.so.6 (0x00007fb7ed506000) /gnu/store/h90vnqw0nwd0hhm1l5dgxsdrigddfmq4-glibc-2.28/lib/ld-linux-x86-64.so.2 => /usr/lib64/ld-linux-x86-64.so.2 (0x00007fb7ee3a0000) ``` And here's what it looks in one of our releases: ``` linux-vdso.so.1 (0x00007ffff52cd000) libpthread.so.0 => /usr/lib/libpthread.so.0 (0x00007f87726b4000) librt.so.1 => /usr/lib/librt.so.1 (0x00007f87726aa000) libm.so.6 => /usr/lib/libm.so.6 (0x00007f8772525000) libgcc_s.so.1 => /usr/lib/libgcc_s.so.1 (0x00007f877250b000) libc.so.6 => /usr/lib/libc.so.6 (0x00007f8772347000) /lib64/ld-linux-x86-64.so.2 => /usr/lib64/ld-linux-x86-64.so.2 (0x00007f8773392000) ``` ~~I suspect it is because my script does not apply the gitian-input patches [described in the release process](https://github.com/bitcoin/bitcoin/blob/master/doc/release-process.md#fetch-and-create-inputs-first-time-or-when-dependency-versions-change) but there is no description as to how these patches are applied.~~ It might also be something else entirely. Edit: It is something else. It appears that the gitian inputs are only used by [`gitian-win-signer.yml`](https://github.com/bitcoin/bitcoin/blob/d6e700e40f861ddd6743f4d13f0d6f6bc19093c2/contrib/gitian-descriptors/gitian-win-signer.yml#L14) </details> ### How to Help 1. Install Guix on your distro either [from source](https://www.gnu.org/software/guix/manual/en/html_node/Requirements.html) or perform a [binary installation](https://www.gnu.org/software/guix/manual/en/html_node/Binary-Installation.html#Binary-Installation) 2. Try out my branch and the command described above! ACKs for top commit: MarcoFalke: Thanks for the replies. ACK 751549b52a9a4cd27389d807ae67f02bbb39cd7f laanwj: ACK 751549b52a9a4cd27389d807ae67f02bbb39cd7f Tree-SHA512: 50e6ab58c6bda9a67125b6271daf7eff0ca57d0efa8941ed3cd951e5bf78b31552fc5e537b1e1bcf2d3cc918c63adf19d685aa117a0f851024dc67e697890a8d
2019-07-12 19:23:30 +02:00
(define* (make-bitcoin-cross-toolchain target
Merge bitcoin/bitcoin#21239: guix: Add codesignature attachment support for osx+win ee883201cf134952284632e9e9ae72bf1c8c792f guix: repro: Sort find output in libtool for gcc-8 (Carl Dong) ee0a67c32a8861eab650bf8894af06807578eba0 codesigning: Use SHA256 as digest for osslsigncode (Windows) (Carl Dong) 38eb91eb0616ed6dbe34c23e11588d130fef07f8 guix: Add codesigning functionality (Carl Dong) bac2690e6f683fcedb883fe1d32f3c33c628a141 guix: Package codesigning tools (Carl Dong) 0a2176d47767972e4cd5ed302c1dbeedece1708b guix: Reindent existing manifest.scm (Carl Dong) c090a3e9238ba2df07875b4708e908d8dca4ed9b Makefile.am: use APP_DIST_DIR instead of hard-coding dist (Carl Dong) Pull request description: This is the last PR before we reach feature-parity with the Gitian process! Note: I tried using the `Makefile` inside the distsrc to make the dmg instead of manually listing out the commands, but `make` seems to want to re-make a lot of other files which broke the dmg. The workflow looks something like this: 1. `env [ FOO=bar... ] ./contrib/guix/guix-build` (add additional env vars as necessary) 2. Codesigners only: 1. Copy `guix-build-<short-id>/output/x86_64-apple-darwin18/bitcoin-<short-id>-osx-unsigned.tar.gz` and `guix-build-<short-id>/output/x86_64-w64-mingw32/bitcoin-<short-id>-win-unsigned.tar.gz` to signing computer 2. Codesign with `./detached-sig-create.sh` inside the tarball 3. Upload contents of `signature-{osx,win}.tar.gz` to https://github.com/bitcoin-core/bitcoin-detached-sigs (as a new tag) 3. Checkout new tag for `bitcoin-core/bitcoin-detached-sigs` with the detached signatures 4. `env [ FOO=bar... ] DETACHED_SIGS_REPO=<path/to/bitcoin-detached-sigs> ./contrib/guix/guix-codesign` (modify env vars as necessary) 5. Make sure `guix.sigs` is cloned and updated 6. `env GUIX_SIGS_REPO=<path/to/guix.sigs> SIGNER=0x96AB007F1A7ED999=dongcarl ./contrib/guix/guix-attest` (modify env vars as necessary) 7. Commit your new signatures and SHA256SUMS in `guix.sigs` 8. Optionally, after there are multiple signatures in `guix.sigs`: `env GUIX_SIGS_REPO=<path/to/guix.sigs> ./contrib/guix/guix-verify` ACKs for top commit: laanwj: Tested ACK ee883201cf134952284632e9e9ae72bf1c8c792f achow101: ACK ee883201cf134952284632e9e9ae72bf1c8c792f Tree-SHA512: e812a07a5f19f900600c70cb9c717769ef544a6c0c12760b5558b76b6b37df863257f3dbf38b0757e6e06e334470267e94c9f2bdbc27409d6837b1a0bfc6acbc
2021-05-24 15:31:05 +02:00
#:key
(base-gcc-for-libc gcc-7)
(base-kernel-headers linux-libre-headers-4.9)
(base-libc (make-glibc-without-ssp glibc-2.24))
Merge bitcoin/bitcoin#21239: guix: Add codesignature attachment support for osx+win ee883201cf134952284632e9e9ae72bf1c8c792f guix: repro: Sort find output in libtool for gcc-8 (Carl Dong) ee0a67c32a8861eab650bf8894af06807578eba0 codesigning: Use SHA256 as digest for osslsigncode (Windows) (Carl Dong) 38eb91eb0616ed6dbe34c23e11588d130fef07f8 guix: Add codesigning functionality (Carl Dong) bac2690e6f683fcedb883fe1d32f3c33c628a141 guix: Package codesigning tools (Carl Dong) 0a2176d47767972e4cd5ed302c1dbeedece1708b guix: Reindent existing manifest.scm (Carl Dong) c090a3e9238ba2df07875b4708e908d8dca4ed9b Makefile.am: use APP_DIST_DIR instead of hard-coding dist (Carl Dong) Pull request description: This is the last PR before we reach feature-parity with the Gitian process! Note: I tried using the `Makefile` inside the distsrc to make the dmg instead of manually listing out the commands, but `make` seems to want to re-make a lot of other files which broke the dmg. The workflow looks something like this: 1. `env [ FOO=bar... ] ./contrib/guix/guix-build` (add additional env vars as necessary) 2. Codesigners only: 1. Copy `guix-build-<short-id>/output/x86_64-apple-darwin18/bitcoin-<short-id>-osx-unsigned.tar.gz` and `guix-build-<short-id>/output/x86_64-w64-mingw32/bitcoin-<short-id>-win-unsigned.tar.gz` to signing computer 2. Codesign with `./detached-sig-create.sh` inside the tarball 3. Upload contents of `signature-{osx,win}.tar.gz` to https://github.com/bitcoin-core/bitcoin-detached-sigs (as a new tag) 3. Checkout new tag for `bitcoin-core/bitcoin-detached-sigs` with the detached signatures 4. `env [ FOO=bar... ] DETACHED_SIGS_REPO=<path/to/bitcoin-detached-sigs> ./contrib/guix/guix-codesign` (modify env vars as necessary) 5. Make sure `guix.sigs` is cloned and updated 6. `env GUIX_SIGS_REPO=<path/to/guix.sigs> SIGNER=0x96AB007F1A7ED999=dongcarl ./contrib/guix/guix-attest` (modify env vars as necessary) 7. Commit your new signatures and SHA256SUMS in `guix.sigs` 8. Optionally, after there are multiple signatures in `guix.sigs`: `env GUIX_SIGS_REPO=<path/to/guix.sigs> ./contrib/guix/guix-verify` ACKs for top commit: laanwj: Tested ACK ee883201cf134952284632e9e9ae72bf1c8c792f achow101: ACK ee883201cf134952284632e9e9ae72bf1c8c792f Tree-SHA512: e812a07a5f19f900600c70cb9c717769ef544a6c0c12760b5558b76b6b37df863257f3dbf38b0757e6e06e334470267e94c9f2bdbc27409d6837b1a0bfc6acbc
2021-05-24 15:31:05 +02:00
(base-gcc (make-gcc-rpath-link base-gcc)))
"Convenience wrapper around MAKE-CROSS-TOOLCHAIN with default values
2023-03-31 12:01:55 +02:00
desirable for building Dash Core release binaries."
Merge #15277: contrib: Enable building in Guix containers 751549b52a9a4cd27389d807ae67f02bbb39cd7f contrib: guix: Additional clarifications re: substitutes (Carl Dong) cd3e947f50db7cfe05c05b368c25742193729a62 contrib: guix: Various improvements. (Carl Dong) 8dff3e48a9e03299468ed3b342642f01f70da9db contrib: guix: Clarify SOURCE_DATE_EPOCH. (Carl Dong) 3e80ec3ea9691c7c89173de922a113e643fe976b contrib: Add deterministic Guix builds. (Carl Dong) Pull request description: ~~**This post is kept updated as this project progresses. Use this [latest update link](https://github.com/bitcoin/bitcoin/pull/15277#issuecomment-497303718) to see what's new.**~~ Please read the `README.md`. ----- ### Guix Introduction This PR enables building bitcoin in Guix containers. [Guix](https://www.gnu.org/software/guix/manual/en/html_node/Features.html) is a transactional package manager much like Nix, but unlike Nix, it has more of a focus on [bootstrappability](https://www.gnu.org/software/guix/manual/en/html_node/Bootstrapping.html) and [reproducibility](https://www.gnu.org/software/guix/blog/tags/reproducible-builds/) which are attractive for security-sensitive projects like bitcoin. ### Guix Build Walkthrough Please read the `README.md`. [Old instructions no. 4](https://github.com/bitcoin/bitcoin/pull/15277#issuecomment-497303718) [Old instructions no. 3](https://github.com/bitcoin/bitcoin/pull/15277#issuecomment-493827011) [Old instructions no. 2](https://github.com/bitcoin/bitcoin/pull/15277#issuecomment-471658439) <details> <summary>Old instructions no. 1</summary> In this PR, we define a Guix [manifest](https://www.gnu.org/software/guix/manual/en/html_node/Invoking-guix-package.html#profile_002dmanifest) in `contrib/guix/manifest.scm`, which declares what packages we want in our environment. We can then invoke ``` guix environment --manifest=contrib/guix/manifest.scm --container --pure --no-grafts --no-substitutes ``` To have Guix: 1. Build an environment containing the packages we defined in our `contrib/guix/manifest.scm` manifest from the Guix bootstrap binaries (see [bootstrappability](https://www.gnu.org/software/guix/manual/en/html_node/Bootstrapping.html) for more details). 2. Start a container with that environment that has no network access, and no access to the host's filesystem except to the `pwd` that it was started in. 3. Drop you into a shell in that container. > Note: if you don't want to wait hours for Guix to build the entire world from scratch, you can eliminate the `--no-substitutes` option to have Guix download from available binary sources. Note that this convenience doesn't necessarily compromise your security, as you can check that a package was built correctly after the fact using `guix build --check <packagename>` Therefore, we can perform a build of bitcoin much like in Gitian by invoking the following: ``` make -C depends -j"$(nproc)" download && \ cat contrib/guix/build.sh | guix environment --manifest=contrib/guix/manifest.scm --container --pure --no-grafts --no-substitutes ``` We don't include `make -C depends -j"$(nproc)" download` inside `contrib/guix/build.sh` because `contrib/guix/build.sh` is run inside the container, which has no network access (which is a good thing). </details> ### Rationale I believe that this represents a substantial improvement for the "supply chain security" of bitcoin because: 1. We no longer have to rely on Ubuntu for our build environment for our releases ([oh the horror](https://github.com/bitcoin/bitcoin/blob/72bd4ab867e3be0d8410403d9641c08288d343e3/contrib/gitian-descriptors/gitian-linux.yml#L10)), because Guix builds everything about the container, we can perform this on almost any Linux distro/system. 2. It is now much easier to determine what trusted binaries are in our supply chain, and even make a nice visualization! (see [bootstrappability](https://www.gnu.org/software/guix/manual/en/html_node/Bootstrapping.html)). 3. There is active effort among Guix folks to minimize the number of trusted binaries even further. OriansJ's [stage0](https://github.com/oriansj/stage0), and janneke's [Mes](https://www.gnu.org/software/mes/) all aim to achieve [reduced binary boostrap](http://joyofsource.com/reduced-binary-seed-bootstrap.html) for Guix. In fact, I believe if OriansJ gets his way, we will end up some day with only a single trusted binary: hex0 (a ~500 byte self-hosting hex assembler). ### Steps to Completion - [x] Successfully build bitcoin inside the Guix environment - [x] Make `check-symbols` pass - [x] Do the above but without nasty hacks - [x] Solve some of the more innocuous hacks - [ ] Make it cross-compile (HELP WANTED HERE) - [x] Linux - [x] x86_64-linux-gnu - [x] i686-linux-gnu - [x] aarch64-linux-gnu - [x] arm-linux-gnueabihf - [x] riscv64-linux-gnu - [ ] OS X - [ ] x86_64-apple-darwin14 - [ ] Windows - [ ] x86_64-w64-mingw32 - [ ] Maybe make importer for depends syntax - [ ] Document build process for future releases - [ ] Extra: Pin the revision of Guix that we build with with Guix [inferiors](https://www.gnu.org/software/guix/manual/en/html_node/Inferiors.html) ### Help Wanted [Old content no. 3](https://github.com/bitcoin/bitcoin/pull/15277#issuecomment-483318210) [Old content no. 2](https://github.com/bitcoin/bitcoin/pull/15277#issuecomment-471658439) <details> <summary>Old content no. 1</summary> As of now, the command described above to perform a build of bitcoin a lot like Gitian works, but fails at the `check-symbols` stage. This is because a few dynamic libraries are linked in that shouldn't be. Here's what `ldd src/bitcoind` looks like when built in a Guix container: ``` linux-vdso.so.1 (0x00007ffcc2d90000) libdl.so.2 => /gnu/store/h90vnqw0nwd0hhm1l5dgxsdrigddfmq4-glibc-2.28/lib/libdl.so.2 (0x00007fb7eda09000) librt.so.1 => /gnu/store/h90vnqw0nwd0hhm1l5dgxsdrigddfmq4-glibc-2.28/lib/librt.so.1 (0x00007fb7ed9ff000) libstdc++.so.6 => /gnu/store/4sqps8dczv3g7rwbdibfz6rf5jlk7w90-gcc-5.5.0-lib/lib/libstdc++.so.6 (0x00007fb7ed87c000) libpthread.so.0 => /gnu/store/h90vnqw0nwd0hhm1l5dgxsdrigddfmq4-glibc-2.28/lib/libpthread.so.0 (0x00007fb7ed85b000) libm.so.6 => /gnu/store/h90vnqw0nwd0hhm1l5dgxsdrigddfmq4-glibc-2.28/lib/libm.so.6 (0x00007fb7ed6da000) libgcc_s.so.1 => /gnu/store/4sqps8dczv3g7rwbdibfz6rf5jlk7w90-gcc-5.5.0-lib/lib/libgcc_s.so.1 (0x00007fb7ed6bf000) libc.so.6 => /gnu/store/h90vnqw0nwd0hhm1l5dgxsdrigddfmq4-glibc-2.28/lib/libc.so.6 (0x00007fb7ed506000) /gnu/store/h90vnqw0nwd0hhm1l5dgxsdrigddfmq4-glibc-2.28/lib/ld-linux-x86-64.so.2 => /usr/lib64/ld-linux-x86-64.so.2 (0x00007fb7ee3a0000) ``` And here's what it looks in one of our releases: ``` linux-vdso.so.1 (0x00007ffff52cd000) libpthread.so.0 => /usr/lib/libpthread.so.0 (0x00007f87726b4000) librt.so.1 => /usr/lib/librt.so.1 (0x00007f87726aa000) libm.so.6 => /usr/lib/libm.so.6 (0x00007f8772525000) libgcc_s.so.1 => /usr/lib/libgcc_s.so.1 (0x00007f877250b000) libc.so.6 => /usr/lib/libc.so.6 (0x00007f8772347000) /lib64/ld-linux-x86-64.so.2 => /usr/lib64/ld-linux-x86-64.so.2 (0x00007f8773392000) ``` ~~I suspect it is because my script does not apply the gitian-input patches [described in the release process](https://github.com/bitcoin/bitcoin/blob/master/doc/release-process.md#fetch-and-create-inputs-first-time-or-when-dependency-versions-change) but there is no description as to how these patches are applied.~~ It might also be something else entirely. Edit: It is something else. It appears that the gitian inputs are only used by [`gitian-win-signer.yml`](https://github.com/bitcoin/bitcoin/blob/d6e700e40f861ddd6743f4d13f0d6f6bc19093c2/contrib/gitian-descriptors/gitian-win-signer.yml#L14) </details> ### How to Help 1. Install Guix on your distro either [from source](https://www.gnu.org/software/guix/manual/en/html_node/Requirements.html) or perform a [binary installation](https://www.gnu.org/software/guix/manual/en/html_node/Binary-Installation.html#Binary-Installation) 2. Try out my branch and the command described above! ACKs for top commit: MarcoFalke: Thanks for the replies. ACK 751549b52a9a4cd27389d807ae67f02bbb39cd7f laanwj: ACK 751549b52a9a4cd27389d807ae67f02bbb39cd7f Tree-SHA512: 50e6ab58c6bda9a67125b6271daf7eff0ca57d0efa8941ed3cd951e5bf78b31552fc5e537b1e1bcf2d3cc918c63adf19d685aa117a0f851024dc67e697890a8d
2019-07-12 19:23:30 +02:00
(make-cross-toolchain target
Merge bitcoin/bitcoin#21239: guix: Add codesignature attachment support for osx+win ee883201cf134952284632e9e9ae72bf1c8c792f guix: repro: Sort find output in libtool for gcc-8 (Carl Dong) ee0a67c32a8861eab650bf8894af06807578eba0 codesigning: Use SHA256 as digest for osslsigncode (Windows) (Carl Dong) 38eb91eb0616ed6dbe34c23e11588d130fef07f8 guix: Add codesigning functionality (Carl Dong) bac2690e6f683fcedb883fe1d32f3c33c628a141 guix: Package codesigning tools (Carl Dong) 0a2176d47767972e4cd5ed302c1dbeedece1708b guix: Reindent existing manifest.scm (Carl Dong) c090a3e9238ba2df07875b4708e908d8dca4ed9b Makefile.am: use APP_DIST_DIR instead of hard-coding dist (Carl Dong) Pull request description: This is the last PR before we reach feature-parity with the Gitian process! Note: I tried using the `Makefile` inside the distsrc to make the dmg instead of manually listing out the commands, but `make` seems to want to re-make a lot of other files which broke the dmg. The workflow looks something like this: 1. `env [ FOO=bar... ] ./contrib/guix/guix-build` (add additional env vars as necessary) 2. Codesigners only: 1. Copy `guix-build-<short-id>/output/x86_64-apple-darwin18/bitcoin-<short-id>-osx-unsigned.tar.gz` and `guix-build-<short-id>/output/x86_64-w64-mingw32/bitcoin-<short-id>-win-unsigned.tar.gz` to signing computer 2. Codesign with `./detached-sig-create.sh` inside the tarball 3. Upload contents of `signature-{osx,win}.tar.gz` to https://github.com/bitcoin-core/bitcoin-detached-sigs (as a new tag) 3. Checkout new tag for `bitcoin-core/bitcoin-detached-sigs` with the detached signatures 4. `env [ FOO=bar... ] DETACHED_SIGS_REPO=<path/to/bitcoin-detached-sigs> ./contrib/guix/guix-codesign` (modify env vars as necessary) 5. Make sure `guix.sigs` is cloned and updated 6. `env GUIX_SIGS_REPO=<path/to/guix.sigs> SIGNER=0x96AB007F1A7ED999=dongcarl ./contrib/guix/guix-attest` (modify env vars as necessary) 7. Commit your new signatures and SHA256SUMS in `guix.sigs` 8. Optionally, after there are multiple signatures in `guix.sigs`: `env GUIX_SIGS_REPO=<path/to/guix.sigs> ./contrib/guix/guix-verify` ACKs for top commit: laanwj: Tested ACK ee883201cf134952284632e9e9ae72bf1c8c792f achow101: ACK ee883201cf134952284632e9e9ae72bf1c8c792f Tree-SHA512: e812a07a5f19f900600c70cb9c717769ef544a6c0c12760b5558b76b6b37df863257f3dbf38b0757e6e06e334470267e94c9f2bdbc27409d6837b1a0bfc6acbc
2021-05-24 15:31:05 +02:00
base-gcc-for-libc
base-kernel-headers
base-libc
base-gcc))
Merge #15277: contrib: Enable building in Guix containers 751549b52a9a4cd27389d807ae67f02bbb39cd7f contrib: guix: Additional clarifications re: substitutes (Carl Dong) cd3e947f50db7cfe05c05b368c25742193729a62 contrib: guix: Various improvements. (Carl Dong) 8dff3e48a9e03299468ed3b342642f01f70da9db contrib: guix: Clarify SOURCE_DATE_EPOCH. (Carl Dong) 3e80ec3ea9691c7c89173de922a113e643fe976b contrib: Add deterministic Guix builds. (Carl Dong) Pull request description: ~~**This post is kept updated as this project progresses. Use this [latest update link](https://github.com/bitcoin/bitcoin/pull/15277#issuecomment-497303718) to see what's new.**~~ Please read the `README.md`. ----- ### Guix Introduction This PR enables building bitcoin in Guix containers. [Guix](https://www.gnu.org/software/guix/manual/en/html_node/Features.html) is a transactional package manager much like Nix, but unlike Nix, it has more of a focus on [bootstrappability](https://www.gnu.org/software/guix/manual/en/html_node/Bootstrapping.html) and [reproducibility](https://www.gnu.org/software/guix/blog/tags/reproducible-builds/) which are attractive for security-sensitive projects like bitcoin. ### Guix Build Walkthrough Please read the `README.md`. [Old instructions no. 4](https://github.com/bitcoin/bitcoin/pull/15277#issuecomment-497303718) [Old instructions no. 3](https://github.com/bitcoin/bitcoin/pull/15277#issuecomment-493827011) [Old instructions no. 2](https://github.com/bitcoin/bitcoin/pull/15277#issuecomment-471658439) <details> <summary>Old instructions no. 1</summary> In this PR, we define a Guix [manifest](https://www.gnu.org/software/guix/manual/en/html_node/Invoking-guix-package.html#profile_002dmanifest) in `contrib/guix/manifest.scm`, which declares what packages we want in our environment. We can then invoke ``` guix environment --manifest=contrib/guix/manifest.scm --container --pure --no-grafts --no-substitutes ``` To have Guix: 1. Build an environment containing the packages we defined in our `contrib/guix/manifest.scm` manifest from the Guix bootstrap binaries (see [bootstrappability](https://www.gnu.org/software/guix/manual/en/html_node/Bootstrapping.html) for more details). 2. Start a container with that environment that has no network access, and no access to the host's filesystem except to the `pwd` that it was started in. 3. Drop you into a shell in that container. > Note: if you don't want to wait hours for Guix to build the entire world from scratch, you can eliminate the `--no-substitutes` option to have Guix download from available binary sources. Note that this convenience doesn't necessarily compromise your security, as you can check that a package was built correctly after the fact using `guix build --check <packagename>` Therefore, we can perform a build of bitcoin much like in Gitian by invoking the following: ``` make -C depends -j"$(nproc)" download && \ cat contrib/guix/build.sh | guix environment --manifest=contrib/guix/manifest.scm --container --pure --no-grafts --no-substitutes ``` We don't include `make -C depends -j"$(nproc)" download` inside `contrib/guix/build.sh` because `contrib/guix/build.sh` is run inside the container, which has no network access (which is a good thing). </details> ### Rationale I believe that this represents a substantial improvement for the "supply chain security" of bitcoin because: 1. We no longer have to rely on Ubuntu for our build environment for our releases ([oh the horror](https://github.com/bitcoin/bitcoin/blob/72bd4ab867e3be0d8410403d9641c08288d343e3/contrib/gitian-descriptors/gitian-linux.yml#L10)), because Guix builds everything about the container, we can perform this on almost any Linux distro/system. 2. It is now much easier to determine what trusted binaries are in our supply chain, and even make a nice visualization! (see [bootstrappability](https://www.gnu.org/software/guix/manual/en/html_node/Bootstrapping.html)). 3. There is active effort among Guix folks to minimize the number of trusted binaries even further. OriansJ's [stage0](https://github.com/oriansj/stage0), and janneke's [Mes](https://www.gnu.org/software/mes/) all aim to achieve [reduced binary boostrap](http://joyofsource.com/reduced-binary-seed-bootstrap.html) for Guix. In fact, I believe if OriansJ gets his way, we will end up some day with only a single trusted binary: hex0 (a ~500 byte self-hosting hex assembler). ### Steps to Completion - [x] Successfully build bitcoin inside the Guix environment - [x] Make `check-symbols` pass - [x] Do the above but without nasty hacks - [x] Solve some of the more innocuous hacks - [ ] Make it cross-compile (HELP WANTED HERE) - [x] Linux - [x] x86_64-linux-gnu - [x] i686-linux-gnu - [x] aarch64-linux-gnu - [x] arm-linux-gnueabihf - [x] riscv64-linux-gnu - [ ] OS X - [ ] x86_64-apple-darwin14 - [ ] Windows - [ ] x86_64-w64-mingw32 - [ ] Maybe make importer for depends syntax - [ ] Document build process for future releases - [ ] Extra: Pin the revision of Guix that we build with with Guix [inferiors](https://www.gnu.org/software/guix/manual/en/html_node/Inferiors.html) ### Help Wanted [Old content no. 3](https://github.com/bitcoin/bitcoin/pull/15277#issuecomment-483318210) [Old content no. 2](https://github.com/bitcoin/bitcoin/pull/15277#issuecomment-471658439) <details> <summary>Old content no. 1</summary> As of now, the command described above to perform a build of bitcoin a lot like Gitian works, but fails at the `check-symbols` stage. This is because a few dynamic libraries are linked in that shouldn't be. Here's what `ldd src/bitcoind` looks like when built in a Guix container: ``` linux-vdso.so.1 (0x00007ffcc2d90000) libdl.so.2 => /gnu/store/h90vnqw0nwd0hhm1l5dgxsdrigddfmq4-glibc-2.28/lib/libdl.so.2 (0x00007fb7eda09000) librt.so.1 => /gnu/store/h90vnqw0nwd0hhm1l5dgxsdrigddfmq4-glibc-2.28/lib/librt.so.1 (0x00007fb7ed9ff000) libstdc++.so.6 => /gnu/store/4sqps8dczv3g7rwbdibfz6rf5jlk7w90-gcc-5.5.0-lib/lib/libstdc++.so.6 (0x00007fb7ed87c000) libpthread.so.0 => /gnu/store/h90vnqw0nwd0hhm1l5dgxsdrigddfmq4-glibc-2.28/lib/libpthread.so.0 (0x00007fb7ed85b000) libm.so.6 => /gnu/store/h90vnqw0nwd0hhm1l5dgxsdrigddfmq4-glibc-2.28/lib/libm.so.6 (0x00007fb7ed6da000) libgcc_s.so.1 => /gnu/store/4sqps8dczv3g7rwbdibfz6rf5jlk7w90-gcc-5.5.0-lib/lib/libgcc_s.so.1 (0x00007fb7ed6bf000) libc.so.6 => /gnu/store/h90vnqw0nwd0hhm1l5dgxsdrigddfmq4-glibc-2.28/lib/libc.so.6 (0x00007fb7ed506000) /gnu/store/h90vnqw0nwd0hhm1l5dgxsdrigddfmq4-glibc-2.28/lib/ld-linux-x86-64.so.2 => /usr/lib64/ld-linux-x86-64.so.2 (0x00007fb7ee3a0000) ``` And here's what it looks in one of our releases: ``` linux-vdso.so.1 (0x00007ffff52cd000) libpthread.so.0 => /usr/lib/libpthread.so.0 (0x00007f87726b4000) librt.so.1 => /usr/lib/librt.so.1 (0x00007f87726aa000) libm.so.6 => /usr/lib/libm.so.6 (0x00007f8772525000) libgcc_s.so.1 => /usr/lib/libgcc_s.so.1 (0x00007f877250b000) libc.so.6 => /usr/lib/libc.so.6 (0x00007f8772347000) /lib64/ld-linux-x86-64.so.2 => /usr/lib64/ld-linux-x86-64.so.2 (0x00007f8773392000) ``` ~~I suspect it is because my script does not apply the gitian-input patches [described in the release process](https://github.com/bitcoin/bitcoin/blob/master/doc/release-process.md#fetch-and-create-inputs-first-time-or-when-dependency-versions-change) but there is no description as to how these patches are applied.~~ It might also be something else entirely. Edit: It is something else. It appears that the gitian inputs are only used by [`gitian-win-signer.yml`](https://github.com/bitcoin/bitcoin/blob/d6e700e40f861ddd6743f4d13f0d6f6bc19093c2/contrib/gitian-descriptors/gitian-win-signer.yml#L14) </details> ### How to Help 1. Install Guix on your distro either [from source](https://www.gnu.org/software/guix/manual/en/html_node/Requirements.html) or perform a [binary installation](https://www.gnu.org/software/guix/manual/en/html_node/Binary-Installation.html#Binary-Installation) 2. Try out my branch and the command described above! ACKs for top commit: MarcoFalke: Thanks for the replies. ACK 751549b52a9a4cd27389d807ae67f02bbb39cd7f laanwj: ACK 751549b52a9a4cd27389d807ae67f02bbb39cd7f Tree-SHA512: 50e6ab58c6bda9a67125b6271daf7eff0ca57d0efa8941ed3cd951e5bf78b31552fc5e537b1e1bcf2d3cc918c63adf19d685aa117a0f851024dc67e697890a8d
2019-07-12 19:23:30 +02:00
Merge #17595: guix: Enable building for `x86_64-w64-mingw32` target a35e3235891d35daa167116cc70340140e883f06 guix: Appease travis. (Carl Dong) 0b66d22da5f53640e22f05adf880782c613e6d0f guix: Use gcc-9 for mingw-w64 instead of 8 (Carl Dong) ba0b99bdd613ba7f17c6247ece3001e1b44759a3 guix: Don't set MINGW_HAS_SECURE_API CFLAG in depends (Carl Dong) 93439a71eda49fb69f1e82966a23a946733aa6fa guix: Bump to upstream commit with mingw-w64 changes (Carl Dong) 35a96792dda9e78165b1598aeac7b2ab759e7be5 guix: Check mingw symbols, improve SSP fix docs (Carl Dong) 449d8fe25bbe25daacfc67aa89ca32b0a3254c5a guix: Expand on INT trap message (Carl Dong) 3f1f03c67a8e9edf487f08d272adb18b0a3942c8 guix: Spelling fixes (Carl Dong) ff821dd2a1c600488d11e7d9a20e9179ecc9144b guix: Reinstate make-ssp-fixed-gcc (Carl Dong) 360a9e0ad50a36ec79a1a160dbed3966689fd41c guix: Bump time-machine for mingw-w64 patches (Carl Dong) 93e41b7e3b54c17fd1b4c61ee95fc0dc2827e954 guix: Use gcc-8 for mingw-w64 instead of 7 (Carl Dong) ef4f7e4c45c60a69406134122f091c77c6ef740f guix: Set the well-known timezone env var (Carl Dong) acf4b3b3b5accf60a19441a0298ef27001b78e72 guix: Make x86_64-w64-mingw32 builds reproducible (Carl Dong) c4cce00eac691625b78b92f7dba0b7f57def19e5 guix: Remove dead links from README. (Carl Dong) df953a4c9a6143f45864757b706c88b6fa70545a guix: Appease shellcheck. (Carl Dong) 91897c95e191d293eb27d8af15cbeafc5b8f3895 guix: Improve guix-build.sh documentation (Carl Dong) 570d769c6c59b9f6d1a2b95b2ed60432cb33b3ba guix: Build support for Windows (Carl Dong) Pull request description: ~~Based on: https://github.com/bitcoin/bitcoin/pull/16519~~ Based on: #17933 (Time Machines are... shall we say... superior :grin:) This PR allows us to perform Guix builds for the `x86_64-w64-mingw32` target. We do this _without_ splitting up the build script like we do in Gitian by using this newfangled alien technology called `case` statements. (This is WIP and might be changed to `if` statements soon) ACKs for top commit: fanquake: ACK a35e3235891d35daa167116cc70340140e883f06 2/3 Tree-SHA512: c471951c23eb2cda919a71285d8b8f2580cb20f09d5db17b53e13dbd8813e01b3e7a83ea848e4913fd0f2bc12c6c133c5f76b54e65c0d89fed4dfd2e0be19875
2020-04-12 12:08:46 +02:00
(define (make-gcc-with-pthreads gcc)
(package-with-extra-configure-variable gcc "--enable-threads" "posix"))
(define (make-mingw-pthreads-cross-toolchain target)
"Create a cross-compilation toolchain package for TARGET"
(let* ((xbinutils (make-binutils-with-mingw-w64-disable-flags (cross-binutils target)))
Merge #17595: guix: Enable building for `x86_64-w64-mingw32` target a35e3235891d35daa167116cc70340140e883f06 guix: Appease travis. (Carl Dong) 0b66d22da5f53640e22f05adf880782c613e6d0f guix: Use gcc-9 for mingw-w64 instead of 8 (Carl Dong) ba0b99bdd613ba7f17c6247ece3001e1b44759a3 guix: Don't set MINGW_HAS_SECURE_API CFLAG in depends (Carl Dong) 93439a71eda49fb69f1e82966a23a946733aa6fa guix: Bump to upstream commit with mingw-w64 changes (Carl Dong) 35a96792dda9e78165b1598aeac7b2ab759e7be5 guix: Check mingw symbols, improve SSP fix docs (Carl Dong) 449d8fe25bbe25daacfc67aa89ca32b0a3254c5a guix: Expand on INT trap message (Carl Dong) 3f1f03c67a8e9edf487f08d272adb18b0a3942c8 guix: Spelling fixes (Carl Dong) ff821dd2a1c600488d11e7d9a20e9179ecc9144b guix: Reinstate make-ssp-fixed-gcc (Carl Dong) 360a9e0ad50a36ec79a1a160dbed3966689fd41c guix: Bump time-machine for mingw-w64 patches (Carl Dong) 93e41b7e3b54c17fd1b4c61ee95fc0dc2827e954 guix: Use gcc-8 for mingw-w64 instead of 7 (Carl Dong) ef4f7e4c45c60a69406134122f091c77c6ef740f guix: Set the well-known timezone env var (Carl Dong) acf4b3b3b5accf60a19441a0298ef27001b78e72 guix: Make x86_64-w64-mingw32 builds reproducible (Carl Dong) c4cce00eac691625b78b92f7dba0b7f57def19e5 guix: Remove dead links from README. (Carl Dong) df953a4c9a6143f45864757b706c88b6fa70545a guix: Appease shellcheck. (Carl Dong) 91897c95e191d293eb27d8af15cbeafc5b8f3895 guix: Improve guix-build.sh documentation (Carl Dong) 570d769c6c59b9f6d1a2b95b2ed60432cb33b3ba guix: Build support for Windows (Carl Dong) Pull request description: ~~Based on: https://github.com/bitcoin/bitcoin/pull/16519~~ Based on: #17933 (Time Machines are... shall we say... superior :grin:) This PR allows us to perform Guix builds for the `x86_64-w64-mingw32` target. We do this _without_ splitting up the build script like we do in Gitian by using this newfangled alien technology called `case` statements. (This is WIP and might be changed to `if` statements soon) ACKs for top commit: fanquake: ACK a35e3235891d35daa167116cc70340140e883f06 2/3 Tree-SHA512: c471951c23eb2cda919a71285d8b8f2580cb20f09d5db17b53e13dbd8813e01b3e7a83ea848e4913fd0f2bc12c6c133c5f76b54e65c0d89fed4dfd2e0be19875
2020-04-12 12:08:46 +02:00
(pthreads-xlibc mingw-w64-x86_64-winpthreads)
(pthreads-xgcc (make-gcc-with-pthreads
(cross-gcc target
Merge bitcoin/bitcoin#21239: guix: Add codesignature attachment support for osx+win ee883201cf134952284632e9e9ae72bf1c8c792f guix: repro: Sort find output in libtool for gcc-8 (Carl Dong) ee0a67c32a8861eab650bf8894af06807578eba0 codesigning: Use SHA256 as digest for osslsigncode (Windows) (Carl Dong) 38eb91eb0616ed6dbe34c23e11588d130fef07f8 guix: Add codesigning functionality (Carl Dong) bac2690e6f683fcedb883fe1d32f3c33c628a141 guix: Package codesigning tools (Carl Dong) 0a2176d47767972e4cd5ed302c1dbeedece1708b guix: Reindent existing manifest.scm (Carl Dong) c090a3e9238ba2df07875b4708e908d8dca4ed9b Makefile.am: use APP_DIST_DIR instead of hard-coding dist (Carl Dong) Pull request description: This is the last PR before we reach feature-parity with the Gitian process! Note: I tried using the `Makefile` inside the distsrc to make the dmg instead of manually listing out the commands, but `make` seems to want to re-make a lot of other files which broke the dmg. The workflow looks something like this: 1. `env [ FOO=bar... ] ./contrib/guix/guix-build` (add additional env vars as necessary) 2. Codesigners only: 1. Copy `guix-build-<short-id>/output/x86_64-apple-darwin18/bitcoin-<short-id>-osx-unsigned.tar.gz` and `guix-build-<short-id>/output/x86_64-w64-mingw32/bitcoin-<short-id>-win-unsigned.tar.gz` to signing computer 2. Codesign with `./detached-sig-create.sh` inside the tarball 3. Upload contents of `signature-{osx,win}.tar.gz` to https://github.com/bitcoin-core/bitcoin-detached-sigs (as a new tag) 3. Checkout new tag for `bitcoin-core/bitcoin-detached-sigs` with the detached signatures 4. `env [ FOO=bar... ] DETACHED_SIGS_REPO=<path/to/bitcoin-detached-sigs> ./contrib/guix/guix-codesign` (modify env vars as necessary) 5. Make sure `guix.sigs` is cloned and updated 6. `env GUIX_SIGS_REPO=<path/to/guix.sigs> SIGNER=0x96AB007F1A7ED999=dongcarl ./contrib/guix/guix-attest` (modify env vars as necessary) 7. Commit your new signatures and SHA256SUMS in `guix.sigs` 8. Optionally, after there are multiple signatures in `guix.sigs`: `env GUIX_SIGS_REPO=<path/to/guix.sigs> ./contrib/guix/guix-verify` ACKs for top commit: laanwj: Tested ACK ee883201cf134952284632e9e9ae72bf1c8c792f achow101: ACK ee883201cf134952284632e9e9ae72bf1c8c792f Tree-SHA512: e812a07a5f19f900600c70cb9c717769ef544a6c0c12760b5558b76b6b37df863257f3dbf38b0757e6e06e334470267e94c9f2bdbc27409d6837b1a0bfc6acbc
2021-05-24 15:31:05 +02:00
#:xgcc (make-ssp-fixed-gcc base-gcc)
Merge #17595: guix: Enable building for `x86_64-w64-mingw32` target a35e3235891d35daa167116cc70340140e883f06 guix: Appease travis. (Carl Dong) 0b66d22da5f53640e22f05adf880782c613e6d0f guix: Use gcc-9 for mingw-w64 instead of 8 (Carl Dong) ba0b99bdd613ba7f17c6247ece3001e1b44759a3 guix: Don't set MINGW_HAS_SECURE_API CFLAG in depends (Carl Dong) 93439a71eda49fb69f1e82966a23a946733aa6fa guix: Bump to upstream commit with mingw-w64 changes (Carl Dong) 35a96792dda9e78165b1598aeac7b2ab759e7be5 guix: Check mingw symbols, improve SSP fix docs (Carl Dong) 449d8fe25bbe25daacfc67aa89ca32b0a3254c5a guix: Expand on INT trap message (Carl Dong) 3f1f03c67a8e9edf487f08d272adb18b0a3942c8 guix: Spelling fixes (Carl Dong) ff821dd2a1c600488d11e7d9a20e9179ecc9144b guix: Reinstate make-ssp-fixed-gcc (Carl Dong) 360a9e0ad50a36ec79a1a160dbed3966689fd41c guix: Bump time-machine for mingw-w64 patches (Carl Dong) 93e41b7e3b54c17fd1b4c61ee95fc0dc2827e954 guix: Use gcc-8 for mingw-w64 instead of 7 (Carl Dong) ef4f7e4c45c60a69406134122f091c77c6ef740f guix: Set the well-known timezone env var (Carl Dong) acf4b3b3b5accf60a19441a0298ef27001b78e72 guix: Make x86_64-w64-mingw32 builds reproducible (Carl Dong) c4cce00eac691625b78b92f7dba0b7f57def19e5 guix: Remove dead links from README. (Carl Dong) df953a4c9a6143f45864757b706c88b6fa70545a guix: Appease shellcheck. (Carl Dong) 91897c95e191d293eb27d8af15cbeafc5b8f3895 guix: Improve guix-build.sh documentation (Carl Dong) 570d769c6c59b9f6d1a2b95b2ed60432cb33b3ba guix: Build support for Windows (Carl Dong) Pull request description: ~~Based on: https://github.com/bitcoin/bitcoin/pull/16519~~ Based on: #17933 (Time Machines are... shall we say... superior :grin:) This PR allows us to perform Guix builds for the `x86_64-w64-mingw32` target. We do this _without_ splitting up the build script like we do in Gitian by using this newfangled alien technology called `case` statements. (This is WIP and might be changed to `if` statements soon) ACKs for top commit: fanquake: ACK a35e3235891d35daa167116cc70340140e883f06 2/3 Tree-SHA512: c471951c23eb2cda919a71285d8b8f2580cb20f09d5db17b53e13dbd8813e01b3e7a83ea848e4913fd0f2bc12c6c133c5f76b54e65c0d89fed4dfd2e0be19875
2020-04-12 12:08:46 +02:00
#:xbinutils xbinutils
#:libc pthreads-xlibc))))
;; Define a meta-package that propagates the resulting XBINUTILS, XLIBC, and
;; XGCC
(package
(name (string-append target "-posix-toolchain"))
(version (package-version pthreads-xgcc))
(source #f)
(build-system trivial-build-system)
(arguments '(#:builder (begin (mkdir %output) #t)))
(propagated-inputs
`(("binutils" ,xbinutils)
("libc" ,pthreads-xlibc)
("gcc" ,pthreads-xgcc)
("gcc-lib" ,pthreads-xgcc "lib")))
Merge #17595: guix: Enable building for `x86_64-w64-mingw32` target a35e3235891d35daa167116cc70340140e883f06 guix: Appease travis. (Carl Dong) 0b66d22da5f53640e22f05adf880782c613e6d0f guix: Use gcc-9 for mingw-w64 instead of 8 (Carl Dong) ba0b99bdd613ba7f17c6247ece3001e1b44759a3 guix: Don't set MINGW_HAS_SECURE_API CFLAG in depends (Carl Dong) 93439a71eda49fb69f1e82966a23a946733aa6fa guix: Bump to upstream commit with mingw-w64 changes (Carl Dong) 35a96792dda9e78165b1598aeac7b2ab759e7be5 guix: Check mingw symbols, improve SSP fix docs (Carl Dong) 449d8fe25bbe25daacfc67aa89ca32b0a3254c5a guix: Expand on INT trap message (Carl Dong) 3f1f03c67a8e9edf487f08d272adb18b0a3942c8 guix: Spelling fixes (Carl Dong) ff821dd2a1c600488d11e7d9a20e9179ecc9144b guix: Reinstate make-ssp-fixed-gcc (Carl Dong) 360a9e0ad50a36ec79a1a160dbed3966689fd41c guix: Bump time-machine for mingw-w64 patches (Carl Dong) 93e41b7e3b54c17fd1b4c61ee95fc0dc2827e954 guix: Use gcc-8 for mingw-w64 instead of 7 (Carl Dong) ef4f7e4c45c60a69406134122f091c77c6ef740f guix: Set the well-known timezone env var (Carl Dong) acf4b3b3b5accf60a19441a0298ef27001b78e72 guix: Make x86_64-w64-mingw32 builds reproducible (Carl Dong) c4cce00eac691625b78b92f7dba0b7f57def19e5 guix: Remove dead links from README. (Carl Dong) df953a4c9a6143f45864757b706c88b6fa70545a guix: Appease shellcheck. (Carl Dong) 91897c95e191d293eb27d8af15cbeafc5b8f3895 guix: Improve guix-build.sh documentation (Carl Dong) 570d769c6c59b9f6d1a2b95b2ed60432cb33b3ba guix: Build support for Windows (Carl Dong) Pull request description: ~~Based on: https://github.com/bitcoin/bitcoin/pull/16519~~ Based on: #17933 (Time Machines are... shall we say... superior :grin:) This PR allows us to perform Guix builds for the `x86_64-w64-mingw32` target. We do this _without_ splitting up the build script like we do in Gitian by using this newfangled alien technology called `case` statements. (This is WIP and might be changed to `if` statements soon) ACKs for top commit: fanquake: ACK a35e3235891d35daa167116cc70340140e883f06 2/3 Tree-SHA512: c471951c23eb2cda919a71285d8b8f2580cb20f09d5db17b53e13dbd8813e01b3e7a83ea848e4913fd0f2bc12c6c133c5f76b54e65c0d89fed4dfd2e0be19875
2020-04-12 12:08:46 +02:00
(synopsis (string-append "Complete GCC tool chain for " target))
(description (string-append "This package provides a complete GCC tool
chain for " target " development."))
(home-page (package-home-page pthreads-xgcc))
(license (package-license pthreads-xgcc)))))
(define (make-nsis-with-sde-support base-nsis)
(package-with-extra-patches base-nsis
(search-our-patches "nsis-SConstruct-sde-support.patch")))
(define-public font-tuffy
(package
Merge bitcoin/bitcoin#21239: guix: Add codesignature attachment support for osx+win ee883201cf134952284632e9e9ae72bf1c8c792f guix: repro: Sort find output in libtool for gcc-8 (Carl Dong) ee0a67c32a8861eab650bf8894af06807578eba0 codesigning: Use SHA256 as digest for osslsigncode (Windows) (Carl Dong) 38eb91eb0616ed6dbe34c23e11588d130fef07f8 guix: Add codesigning functionality (Carl Dong) bac2690e6f683fcedb883fe1d32f3c33c628a141 guix: Package codesigning tools (Carl Dong) 0a2176d47767972e4cd5ed302c1dbeedece1708b guix: Reindent existing manifest.scm (Carl Dong) c090a3e9238ba2df07875b4708e908d8dca4ed9b Makefile.am: use APP_DIST_DIR instead of hard-coding dist (Carl Dong) Pull request description: This is the last PR before we reach feature-parity with the Gitian process! Note: I tried using the `Makefile` inside the distsrc to make the dmg instead of manually listing out the commands, but `make` seems to want to re-make a lot of other files which broke the dmg. The workflow looks something like this: 1. `env [ FOO=bar... ] ./contrib/guix/guix-build` (add additional env vars as necessary) 2. Codesigners only: 1. Copy `guix-build-<short-id>/output/x86_64-apple-darwin18/bitcoin-<short-id>-osx-unsigned.tar.gz` and `guix-build-<short-id>/output/x86_64-w64-mingw32/bitcoin-<short-id>-win-unsigned.tar.gz` to signing computer 2. Codesign with `./detached-sig-create.sh` inside the tarball 3. Upload contents of `signature-{osx,win}.tar.gz` to https://github.com/bitcoin-core/bitcoin-detached-sigs (as a new tag) 3. Checkout new tag for `bitcoin-core/bitcoin-detached-sigs` with the detached signatures 4. `env [ FOO=bar... ] DETACHED_SIGS_REPO=<path/to/bitcoin-detached-sigs> ./contrib/guix/guix-codesign` (modify env vars as necessary) 5. Make sure `guix.sigs` is cloned and updated 6. `env GUIX_SIGS_REPO=<path/to/guix.sigs> SIGNER=0x96AB007F1A7ED999=dongcarl ./contrib/guix/guix-attest` (modify env vars as necessary) 7. Commit your new signatures and SHA256SUMS in `guix.sigs` 8. Optionally, after there are multiple signatures in `guix.sigs`: `env GUIX_SIGS_REPO=<path/to/guix.sigs> ./contrib/guix/guix-verify` ACKs for top commit: laanwj: Tested ACK ee883201cf134952284632e9e9ae72bf1c8c792f achow101: ACK ee883201cf134952284632e9e9ae72bf1c8c792f Tree-SHA512: e812a07a5f19f900600c70cb9c717769ef544a6c0c12760b5558b76b6b37df863257f3dbf38b0757e6e06e334470267e94c9f2bdbc27409d6837b1a0bfc6acbc
2021-05-24 15:31:05 +02:00
(name "font-tuffy")
(version "20120614")
(source
(origin
(method url-fetch)
(uri (string-append "http://tulrich.com/fonts/tuffy-" version ".tar.gz"))
(file-name (string-append name "-" version ".tar.gz"))
(sha256
(base32
"02vf72bgrp30vrbfhxjw82s115z27dwfgnmmzfb0n9wfhxxfpyf6"))))
(build-system font-build-system)
(home-page "http://tulrich.com/fonts/")
(synopsis "The Tuffy Truetype Font Family")
(description
"Thatcher Ulrich's first outline font design. He started with the goal of producing a neutral, readable sans-serif text font. There are lots of \"expressive\" fonts out there, but he wanted to start with something very plain and clean, something he might want to actually use. ")
(license license:public-domain)))
(define osslsigncode
(package
(name "osslsigncode")
(version "2.0")
(source (origin
(method url-fetch)
(uri (string-append "https://github.com/mtrojnar/"
name "/archive/" version ".tar.gz"))
(sha256
(base32
"0byri6xny770wwb2nciq44j5071122l14bvv65axdd70nfjf0q2s"))))
(build-system gnu-build-system)
(native-inputs
`(("pkg-config" ,pkg-config)
("autoconf" ,autoconf)
("automake" ,automake)
("libtool" ,libtool)))
(inputs
`(("openssl" ,openssl)))
(arguments
`(#:configure-flags
`("--without-gsf"
"--without-curl"
"--disable-dependency-tracking")))
(home-page "https://github.com/mtrojnar/osslsigncode")
(synopsis "Authenticode signing and timestamping tool")
(description "osslsigncode is a small tool that implements part of the
functionality of the Microsoft tool signtool.exe - more exactly the Authenticode
signing and timestamping. But osslsigncode is based on OpenSSL and cURL, and
thus should be able to compile on most platforms where these exist.")
(license license:gpl3+))) ; license is with openssl exception
(define-public python-asn1crypto
(package
(name "python-asn1crypto")
(version "1.4.0")
(source
(origin
(method git-fetch)
(uri (git-reference
(url "https://github.com/wbond/asn1crypto")
(commit version)))
(file-name (git-file-name name version))
(sha256
(base32
"19abibn6jw20mzi1ln4n9jjvpdka8ygm4m439hplyrdfqbvgm01r"))))
(build-system python-build-system)
(arguments
'(#:phases
(modify-phases %standard-phases
(replace 'check
(lambda _
(invoke "python" "run.py" "tests"))))))
(home-page "https://github.com/wbond/asn1crypto")
(synopsis "ASN.1 parser and serializer in Python")
(description "asn1crypto is an ASN.1 parser and serializer with definitions
for private keys, public keys, certificates, CRL, OCSP, CMS, PKCS#3, PKCS#7,
PKCS#8, PKCS#12, PKCS#5, X.509 and TSP.")
(license license:expat)))
(define-public python-elfesteem
(let ((commit "87bbd79ab7e361004c98cc8601d4e5f029fd8bd5"))
(package
(name "python-elfesteem")
(version (git-version "0.1" "1" commit))
(source
(origin
(method git-fetch)
(uri (git-reference
(url "https://github.com/LRGH/elfesteem")
(commit commit)))
(file-name (git-file-name name commit))
(sha256
(base32
"1nyvjisvyxyxnd0023xjf5846xd03lwawp5pfzr8vrky7wwm5maz"))))
(build-system python-build-system)
;; There are no tests, but attempting to run python setup.py test leads to
;; PYTHONPATH problems, just disable the test
(arguments '(#:tests? #f))
(home-page "https://github.com/LRGH/elfesteem")
(synopsis "ELF/PE/Mach-O parsing library")
(description "elfesteem parses ELF, PE and Mach-O files.")
(license license:lgpl2.1))))
(define-public python-oscrypto
(package
(name "python-oscrypto")
(version "1.2.1")
(source
(origin
(method git-fetch)
(uri (git-reference
(url "https://github.com/wbond/oscrypto")
(commit version)))
(file-name (git-file-name name version))
(sha256
(base32
"1d4d8s4z340qhvb3g5m5v3436y3a71yc26wk4749q64m09kxqc3l"))
(patches (search-our-patches "oscrypto-hard-code-openssl.patch"))))
(build-system python-build-system)
(native-search-paths
(list (search-path-specification
(variable "SSL_CERT_FILE")
(file-type 'regular)
(separator #f) ;single entry
(files '("etc/ssl/certs/ca-certificates.crt")))))
(propagated-inputs
`(("python-asn1crypto" ,python-asn1crypto)
("openssl" ,openssl)))
(arguments
`(#:phases
(modify-phases %standard-phases
(add-after 'unpack 'hard-code-path-to-libscrypt
(lambda* (#:key inputs #:allow-other-keys)
(let ((openssl (assoc-ref inputs "openssl")))
(substitute* "oscrypto/__init__.py"
(("@GUIX_OSCRYPTO_USE_OPENSSL@")
(string-append openssl "/lib/libcrypto.so" "," openssl "/lib/libssl.so")))
#t)))
(add-after 'unpack 'disable-broken-tests
(lambda _
;; This test is broken as there is no keyboard interrupt.
(substitute* "tests/test_trust_list.py"
(("^(.*)class TrustListTests" line indent)
(string-append indent
"@unittest.skip(\"Disabled by Guix\")\n"
line)))
(substitute* "tests/test_tls.py"
(("^(.*)class TLSTests" line indent)
(string-append indent
"@unittest.skip(\"Disabled by Guix\")\n"
line)))
#t))
(replace 'check
(lambda _
(invoke "python" "run.py" "tests")
#t)))))
(home-page "https://github.com/wbond/oscrypto")
(synopsis "Compiler-free Python crypto library backed by the OS")
(description "oscrypto is a compilation-free, always up-to-date encryption library for Python.")
(license license:expat)))
(define-public python-oscryptotests
(package (inherit python-oscrypto)
(name "python-oscryptotests")
(arguments
`(#:tests? #f
#:phases
(modify-phases %standard-phases
(add-after 'unpack 'hard-code-path-to-libscrypt
(lambda* (#:key inputs #:allow-other-keys)
(chdir "tests")
#t)))))))
(define-public python-certvalidator
(let ((commit "e5bdb4bfcaa09fa0af355eb8867d00dfeecba08c"))
(package
(name "python-certvalidator")
(version (git-version "0.1" "1" commit))
(source
(origin
(method git-fetch)
(uri (git-reference
(url "https://github.com/achow101/certvalidator")
(commit commit)))
(file-name (git-file-name name commit))
(sha256
(base32
"18pvxkvpkfkzgvfylv0kx65pmxfcv1hpsg03cip93krfvrrl4c75"))))
(build-system python-build-system)
(propagated-inputs
`(("python-asn1crypto" ,python-asn1crypto)
("python-oscrypto" ,python-oscrypto)
("python-oscryptotests", python-oscryptotests))) ;; certvalidator tests import oscryptotests
(arguments
`(#:phases
(modify-phases %standard-phases
(add-after 'unpack 'disable-broken-tests
(lambda _
(substitute* "tests/test_certificate_validator.py"
(("^(.*)class CertificateValidatorTests" line indent)
(string-append indent
"@unittest.skip(\"Disabled by Guix\")\n"
line)))
(substitute* "tests/test_crl_client.py"
(("^(.*)def test_fetch_crl" line indent)
(string-append indent
"@unittest.skip(\"Disabled by Guix\")\n"
line)))
(substitute* "tests/test_ocsp_client.py"
(("^(.*)def test_fetch_ocsp" line indent)
(string-append indent
"@unittest.skip(\"Disabled by Guix\")\n"
line)))
(substitute* "tests/test_registry.py"
(("^(.*)def test_build_paths" line indent)
(string-append indent
"@unittest.skip(\"Disabled by Guix\")\n"
line)))
(substitute* "tests/test_validate.py"
(("^(.*)def test_revocation_mode_hard" line indent)
(string-append indent
"@unittest.skip(\"Disabled by Guix\")\n"
line)))
Merge bitcoin/bitcoin#24215: [22.x] guix: ignore additional failing certvalidator test b7ecef1ddf0c9f1f53ab220bee2e19a6b8978e34 guix: ignore additioanl failing certvalidator test (fanquake) Pull request description: Backports https://github.com/bitcoin/bitcoin/commit/85885919656a3f606f3d7f208378aabe95f3f62d from #24057 so that from-scratch Guix builds for the Darwin host aren't broken due to a (very recently) expired certificate causing one of the python-certvalidator tests to fail. Kept separate from #23276 because that hasn't gotten review attention, and I don't think we should leave `22.x` Darwin Guix builds broken for any longer than we have to. Fixes #24110. ```bash ====================================================================== ERROR: test_revocation_mode_soft (tests.test_validate.ValidateTests) ---------------------------------------------------------------------- Traceback (most recent call last): File "/tmp/guix-build-python-certvalidator-0.1-1.e5bdb4b.drv-0/source/tests/test_validate.py", line 85, in test_revocation_mode_soft validate_path(context, path) File "/tmp/guix-build-python-certvalidator-0.1-1.e5bdb4b.drv-0/source/tests/../certvalidator/validate.py", line 50, in validate_path return _validate_path(validation_context, path) File "/tmp/guix-build-python-certvalidator-0.1-1.e5bdb4b.drv-0/source/tests/../certvalidator/validate.py", line 358, in _validate_path raise PathValidationError(pretty_message( certvalidator.errors.PathValidationError: The path could not be validated because the end-entity certificate expired 2022-01-14 12:00:00Z ``` Guix Build: ```bash bash-5.1# find guix-build-$(git rev-parse --short=12 HEAD)/output/ -type f -print0 | env LC_ALL=C sort -z | xargs -r0 sha256sum 359755bffecc64b4c005c5cdee3824190f6b1759dbc6c20034476dcc06413959 guix-build-b7ecef1ddf0c/output/dist-archive/bitcoin-b7ecef1ddf0c.tar.gz 0c6700270ec75991d70a97cad77e22cc00553f812edb56c1bac5ef6421f963e1 guix-build-b7ecef1ddf0c/output/x86_64-apple-darwin/SHA256SUMS.part 87d4637a87959a304422550edf87feda3953d7305894154a6a2d413cc0dd2034 guix-build-b7ecef1ddf0c/output/x86_64-apple-darwin/bitcoin-b7ecef1ddf0c-osx-unsigned.dmg 9cabae32689bd5f93e7faaaf341827f1c4069a63ab6f74276564e47819343b6c guix-build-b7ecef1ddf0c/output/x86_64-apple-darwin/bitcoin-b7ecef1ddf0c-osx-unsigned.tar.gz bb5fb113bc022a305e49783d0ba48be90aca61e4a942beeb45206dbc5b91ca6e guix-build-b7ecef1ddf0c/output/x86_64-apple-darwin/bitcoin-b7ecef1ddf0c-x86_64-apple-darwin.tar.gz ``` ACKs for top commit: MarcoFalke: Concept ACK b7ecef1ddf0c9f1f53ab220bee2e19a6b8978e34 Tree-SHA512: 8f761fece405b3b974b9f42ab4ebf8995d3284ce7bfb0556ff8459e1a7d30f8bd2f407cb5651e9fa1094c493148bba7a8918a251b54a83efe12acfaf3d39f350
2022-02-09 10:29:50 +01:00
(substitute* "tests/test_validate.py"
(("^(.*)def test_revocation_mode_soft" line indent)
(string-append indent
"@unittest.skip(\"Disabled by Guix\")\n"
line)))
Merge bitcoin/bitcoin#21239: guix: Add codesignature attachment support for osx+win ee883201cf134952284632e9e9ae72bf1c8c792f guix: repro: Sort find output in libtool for gcc-8 (Carl Dong) ee0a67c32a8861eab650bf8894af06807578eba0 codesigning: Use SHA256 as digest for osslsigncode (Windows) (Carl Dong) 38eb91eb0616ed6dbe34c23e11588d130fef07f8 guix: Add codesigning functionality (Carl Dong) bac2690e6f683fcedb883fe1d32f3c33c628a141 guix: Package codesigning tools (Carl Dong) 0a2176d47767972e4cd5ed302c1dbeedece1708b guix: Reindent existing manifest.scm (Carl Dong) c090a3e9238ba2df07875b4708e908d8dca4ed9b Makefile.am: use APP_DIST_DIR instead of hard-coding dist (Carl Dong) Pull request description: This is the last PR before we reach feature-parity with the Gitian process! Note: I tried using the `Makefile` inside the distsrc to make the dmg instead of manually listing out the commands, but `make` seems to want to re-make a lot of other files which broke the dmg. The workflow looks something like this: 1. `env [ FOO=bar... ] ./contrib/guix/guix-build` (add additional env vars as necessary) 2. Codesigners only: 1. Copy `guix-build-<short-id>/output/x86_64-apple-darwin18/bitcoin-<short-id>-osx-unsigned.tar.gz` and `guix-build-<short-id>/output/x86_64-w64-mingw32/bitcoin-<short-id>-win-unsigned.tar.gz` to signing computer 2. Codesign with `./detached-sig-create.sh` inside the tarball 3. Upload contents of `signature-{osx,win}.tar.gz` to https://github.com/bitcoin-core/bitcoin-detached-sigs (as a new tag) 3. Checkout new tag for `bitcoin-core/bitcoin-detached-sigs` with the detached signatures 4. `env [ FOO=bar... ] DETACHED_SIGS_REPO=<path/to/bitcoin-detached-sigs> ./contrib/guix/guix-codesign` (modify env vars as necessary) 5. Make sure `guix.sigs` is cloned and updated 6. `env GUIX_SIGS_REPO=<path/to/guix.sigs> SIGNER=0x96AB007F1A7ED999=dongcarl ./contrib/guix/guix-attest` (modify env vars as necessary) 7. Commit your new signatures and SHA256SUMS in `guix.sigs` 8. Optionally, after there are multiple signatures in `guix.sigs`: `env GUIX_SIGS_REPO=<path/to/guix.sigs> ./contrib/guix/guix-verify` ACKs for top commit: laanwj: Tested ACK ee883201cf134952284632e9e9ae72bf1c8c792f achow101: ACK ee883201cf134952284632e9e9ae72bf1c8c792f Tree-SHA512: e812a07a5f19f900600c70cb9c717769ef544a6c0c12760b5558b76b6b37df863257f3dbf38b0757e6e06e334470267e94c9f2bdbc27409d6837b1a0bfc6acbc
2021-05-24 15:31:05 +02:00
#t))
(replace 'check
(lambda _
(invoke "python" "run.py" "tests")
#t)))))
(home-page "https://github.com/wbond/certvalidator")
(synopsis "Python library for validating X.509 certificates and paths")
(description "certvalidator is a Python library for validating X.509
certificates or paths. Supports various options, including: validation at a
specific moment in time, whitelisting and revocation checks.")
(license license:expat))))
(define-public python-requests-2.25.1
(package (inherit python-requests)
(version "2.25.1")
(source (origin
(method url-fetch)
(uri (pypi-uri "requests" version))
(sha256
(base32
"015qflyqsgsz09gnar69s6ga74ivq5kch69s4qxz3904m7a3v5r7"))))))
(define-public python-altgraph
(package
(name "python-altgraph")
(version "0.17")
(source
(origin
(method git-fetch)
(uri (git-reference
(url "https://github.com/ronaldoussoren/altgraph")
(commit (string-append "v" version))))
(file-name (git-file-name name version))
(sha256
(base32
"09sm4srvvkw458pn48ga9q7ykr4xlz7q8gh1h9w7nxpf001qgpwb"))))
(build-system python-build-system)
(home-page "https://github.com/ronaldoussoren/altgraph")
(synopsis "Python graph (network) package")
(description "altgraph is a fork of graphlib: a graph (network) package for
constructing graphs, BFS and DFS traversals, topological sort, shortest paths,
etc. with graphviz output.")
(license license:expat)))
(define-public python-macholib
(package
(name "python-macholib")
(version "1.14")
(source
(origin
(method git-fetch)
(uri (git-reference
(url "https://github.com/ronaldoussoren/macholib")
(commit (string-append "v" version))))
(file-name (git-file-name name version))
(sha256
(base32
"0aislnnfsza9wl4f0vp45ivzlc0pzhp9d4r08700slrypn5flg42"))))
(build-system python-build-system)
(propagated-inputs
`(("python-altgraph" ,python-altgraph)))
(arguments
'(#:phases
(modify-phases %standard-phases
(add-after 'unpack 'disable-broken-tests
(lambda _
;; This test is broken as there is no keyboard interrupt.
(substitute* "macholib_tests/test_command_line.py"
(("^(.*)class TestCmdLine" line indent)
(string-append indent
"@unittest.skip(\"Disabled by Guix\")\n"
line)))
(substitute* "macholib_tests/test_dyld.py"
(("^(.*)def test_\\S+_find" line indent)
(string-append indent
"@unittest.skip(\"Disabled by Guix\")\n"
line))
(("^(.*)def testBasic" line indent)
(string-append indent
"@unittest.skip(\"Disabled by Guix\")\n"
line))
)
#t)))))
(home-page "https://github.com/ronaldoussoren/macholib")
(synopsis "Python library for analyzing and editing Mach-O headers")
(description "macholib is a Macho-O header analyzer and editor. It's
typically used as a dependency analysis tool, and also to rewrite dylib
references in Mach-O headers to be @executable_path relative. Though this tool
targets a platform specific file format, it is pure python code that is platform
and endian independent.")
(license license:expat)))
(define-public python-signapple
(let ((commit "b084cbbf44d5330448ffce0c7d118f75781b64bd"))
Merge bitcoin/bitcoin#21239: guix: Add codesignature attachment support for osx+win ee883201cf134952284632e9e9ae72bf1c8c792f guix: repro: Sort find output in libtool for gcc-8 (Carl Dong) ee0a67c32a8861eab650bf8894af06807578eba0 codesigning: Use SHA256 as digest for osslsigncode (Windows) (Carl Dong) 38eb91eb0616ed6dbe34c23e11588d130fef07f8 guix: Add codesigning functionality (Carl Dong) bac2690e6f683fcedb883fe1d32f3c33c628a141 guix: Package codesigning tools (Carl Dong) 0a2176d47767972e4cd5ed302c1dbeedece1708b guix: Reindent existing manifest.scm (Carl Dong) c090a3e9238ba2df07875b4708e908d8dca4ed9b Makefile.am: use APP_DIST_DIR instead of hard-coding dist (Carl Dong) Pull request description: This is the last PR before we reach feature-parity with the Gitian process! Note: I tried using the `Makefile` inside the distsrc to make the dmg instead of manually listing out the commands, but `make` seems to want to re-make a lot of other files which broke the dmg. The workflow looks something like this: 1. `env [ FOO=bar... ] ./contrib/guix/guix-build` (add additional env vars as necessary) 2. Codesigners only: 1. Copy `guix-build-<short-id>/output/x86_64-apple-darwin18/bitcoin-<short-id>-osx-unsigned.tar.gz` and `guix-build-<short-id>/output/x86_64-w64-mingw32/bitcoin-<short-id>-win-unsigned.tar.gz` to signing computer 2. Codesign with `./detached-sig-create.sh` inside the tarball 3. Upload contents of `signature-{osx,win}.tar.gz` to https://github.com/bitcoin-core/bitcoin-detached-sigs (as a new tag) 3. Checkout new tag for `bitcoin-core/bitcoin-detached-sigs` with the detached signatures 4. `env [ FOO=bar... ] DETACHED_SIGS_REPO=<path/to/bitcoin-detached-sigs> ./contrib/guix/guix-codesign` (modify env vars as necessary) 5. Make sure `guix.sigs` is cloned and updated 6. `env GUIX_SIGS_REPO=<path/to/guix.sigs> SIGNER=0x96AB007F1A7ED999=dongcarl ./contrib/guix/guix-attest` (modify env vars as necessary) 7. Commit your new signatures and SHA256SUMS in `guix.sigs` 8. Optionally, after there are multiple signatures in `guix.sigs`: `env GUIX_SIGS_REPO=<path/to/guix.sigs> ./contrib/guix/guix-verify` ACKs for top commit: laanwj: Tested ACK ee883201cf134952284632e9e9ae72bf1c8c792f achow101: ACK ee883201cf134952284632e9e9ae72bf1c8c792f Tree-SHA512: e812a07a5f19f900600c70cb9c717769ef544a6c0c12760b5558b76b6b37df863257f3dbf38b0757e6e06e334470267e94c9f2bdbc27409d6837b1a0bfc6acbc
2021-05-24 15:31:05 +02:00
(package
(name "python-signapple")
(version (git-version "0.1" "1" commit))
(source
(origin
(method git-fetch)
(uri (git-reference
(url "https://github.com/achow101/signapple")
Merge bitcoin/bitcoin#21239: guix: Add codesignature attachment support for osx+win ee883201cf134952284632e9e9ae72bf1c8c792f guix: repro: Sort find output in libtool for gcc-8 (Carl Dong) ee0a67c32a8861eab650bf8894af06807578eba0 codesigning: Use SHA256 as digest for osslsigncode (Windows) (Carl Dong) 38eb91eb0616ed6dbe34c23e11588d130fef07f8 guix: Add codesigning functionality (Carl Dong) bac2690e6f683fcedb883fe1d32f3c33c628a141 guix: Package codesigning tools (Carl Dong) 0a2176d47767972e4cd5ed302c1dbeedece1708b guix: Reindent existing manifest.scm (Carl Dong) c090a3e9238ba2df07875b4708e908d8dca4ed9b Makefile.am: use APP_DIST_DIR instead of hard-coding dist (Carl Dong) Pull request description: This is the last PR before we reach feature-parity with the Gitian process! Note: I tried using the `Makefile` inside the distsrc to make the dmg instead of manually listing out the commands, but `make` seems to want to re-make a lot of other files which broke the dmg. The workflow looks something like this: 1. `env [ FOO=bar... ] ./contrib/guix/guix-build` (add additional env vars as necessary) 2. Codesigners only: 1. Copy `guix-build-<short-id>/output/x86_64-apple-darwin18/bitcoin-<short-id>-osx-unsigned.tar.gz` and `guix-build-<short-id>/output/x86_64-w64-mingw32/bitcoin-<short-id>-win-unsigned.tar.gz` to signing computer 2. Codesign with `./detached-sig-create.sh` inside the tarball 3. Upload contents of `signature-{osx,win}.tar.gz` to https://github.com/bitcoin-core/bitcoin-detached-sigs (as a new tag) 3. Checkout new tag for `bitcoin-core/bitcoin-detached-sigs` with the detached signatures 4. `env [ FOO=bar... ] DETACHED_SIGS_REPO=<path/to/bitcoin-detached-sigs> ./contrib/guix/guix-codesign` (modify env vars as necessary) 5. Make sure `guix.sigs` is cloned and updated 6. `env GUIX_SIGS_REPO=<path/to/guix.sigs> SIGNER=0x96AB007F1A7ED999=dongcarl ./contrib/guix/guix-attest` (modify env vars as necessary) 7. Commit your new signatures and SHA256SUMS in `guix.sigs` 8. Optionally, after there are multiple signatures in `guix.sigs`: `env GUIX_SIGS_REPO=<path/to/guix.sigs> ./contrib/guix/guix-verify` ACKs for top commit: laanwj: Tested ACK ee883201cf134952284632e9e9ae72bf1c8c792f achow101: ACK ee883201cf134952284632e9e9ae72bf1c8c792f Tree-SHA512: e812a07a5f19f900600c70cb9c717769ef544a6c0c12760b5558b76b6b37df863257f3dbf38b0757e6e06e334470267e94c9f2bdbc27409d6837b1a0bfc6acbc
2021-05-24 15:31:05 +02:00
(commit commit)))
(file-name (git-file-name name commit))
(sha256
(base32
"0k7inccl2mzac3wq4asbr0kl8s4cghm8982z54kfascqg45shv01"))))
Merge bitcoin/bitcoin#21239: guix: Add codesignature attachment support for osx+win ee883201cf134952284632e9e9ae72bf1c8c792f guix: repro: Sort find output in libtool for gcc-8 (Carl Dong) ee0a67c32a8861eab650bf8894af06807578eba0 codesigning: Use SHA256 as digest for osslsigncode (Windows) (Carl Dong) 38eb91eb0616ed6dbe34c23e11588d130fef07f8 guix: Add codesigning functionality (Carl Dong) bac2690e6f683fcedb883fe1d32f3c33c628a141 guix: Package codesigning tools (Carl Dong) 0a2176d47767972e4cd5ed302c1dbeedece1708b guix: Reindent existing manifest.scm (Carl Dong) c090a3e9238ba2df07875b4708e908d8dca4ed9b Makefile.am: use APP_DIST_DIR instead of hard-coding dist (Carl Dong) Pull request description: This is the last PR before we reach feature-parity with the Gitian process! Note: I tried using the `Makefile` inside the distsrc to make the dmg instead of manually listing out the commands, but `make` seems to want to re-make a lot of other files which broke the dmg. The workflow looks something like this: 1. `env [ FOO=bar... ] ./contrib/guix/guix-build` (add additional env vars as necessary) 2. Codesigners only: 1. Copy `guix-build-<short-id>/output/x86_64-apple-darwin18/bitcoin-<short-id>-osx-unsigned.tar.gz` and `guix-build-<short-id>/output/x86_64-w64-mingw32/bitcoin-<short-id>-win-unsigned.tar.gz` to signing computer 2. Codesign with `./detached-sig-create.sh` inside the tarball 3. Upload contents of `signature-{osx,win}.tar.gz` to https://github.com/bitcoin-core/bitcoin-detached-sigs (as a new tag) 3. Checkout new tag for `bitcoin-core/bitcoin-detached-sigs` with the detached signatures 4. `env [ FOO=bar... ] DETACHED_SIGS_REPO=<path/to/bitcoin-detached-sigs> ./contrib/guix/guix-codesign` (modify env vars as necessary) 5. Make sure `guix.sigs` is cloned and updated 6. `env GUIX_SIGS_REPO=<path/to/guix.sigs> SIGNER=0x96AB007F1A7ED999=dongcarl ./contrib/guix/guix-attest` (modify env vars as necessary) 7. Commit your new signatures and SHA256SUMS in `guix.sigs` 8. Optionally, after there are multiple signatures in `guix.sigs`: `env GUIX_SIGS_REPO=<path/to/guix.sigs> ./contrib/guix/guix-verify` ACKs for top commit: laanwj: Tested ACK ee883201cf134952284632e9e9ae72bf1c8c792f achow101: ACK ee883201cf134952284632e9e9ae72bf1c8c792f Tree-SHA512: e812a07a5f19f900600c70cb9c717769ef544a6c0c12760b5558b76b6b37df863257f3dbf38b0757e6e06e334470267e94c9f2bdbc27409d6837b1a0bfc6acbc
2021-05-24 15:31:05 +02:00
(build-system python-build-system)
(propagated-inputs
`(("python-asn1crypto" ,python-asn1crypto)
("python-oscrypto" ,python-oscrypto)
("python-certvalidator" ,python-certvalidator)
("python-elfesteem" ,python-elfesteem)
("python-requests" ,python-requests-2.25.1)
("python-macholib" ,python-macholib)
("libcrypto" ,openssl)))
;; There are no tests, but attempting to run python setup.py test leads to
;; problems, just disable the test
(arguments '(#:tests? #f))
(home-page "https://github.com/achow101/signapple")
(synopsis "Mach-O binary signature tool")
(description "signapple is a Python tool for creating, verifying, and
inspecting signatures in Mach-O binaries.")
(license license:expat))))
Merge #17595: guix: Enable building for `x86_64-w64-mingw32` target a35e3235891d35daa167116cc70340140e883f06 guix: Appease travis. (Carl Dong) 0b66d22da5f53640e22f05adf880782c613e6d0f guix: Use gcc-9 for mingw-w64 instead of 8 (Carl Dong) ba0b99bdd613ba7f17c6247ece3001e1b44759a3 guix: Don't set MINGW_HAS_SECURE_API CFLAG in depends (Carl Dong) 93439a71eda49fb69f1e82966a23a946733aa6fa guix: Bump to upstream commit with mingw-w64 changes (Carl Dong) 35a96792dda9e78165b1598aeac7b2ab759e7be5 guix: Check mingw symbols, improve SSP fix docs (Carl Dong) 449d8fe25bbe25daacfc67aa89ca32b0a3254c5a guix: Expand on INT trap message (Carl Dong) 3f1f03c67a8e9edf487f08d272adb18b0a3942c8 guix: Spelling fixes (Carl Dong) ff821dd2a1c600488d11e7d9a20e9179ecc9144b guix: Reinstate make-ssp-fixed-gcc (Carl Dong) 360a9e0ad50a36ec79a1a160dbed3966689fd41c guix: Bump time-machine for mingw-w64 patches (Carl Dong) 93e41b7e3b54c17fd1b4c61ee95fc0dc2827e954 guix: Use gcc-8 for mingw-w64 instead of 7 (Carl Dong) ef4f7e4c45c60a69406134122f091c77c6ef740f guix: Set the well-known timezone env var (Carl Dong) acf4b3b3b5accf60a19441a0298ef27001b78e72 guix: Make x86_64-w64-mingw32 builds reproducible (Carl Dong) c4cce00eac691625b78b92f7dba0b7f57def19e5 guix: Remove dead links from README. (Carl Dong) df953a4c9a6143f45864757b706c88b6fa70545a guix: Appease shellcheck. (Carl Dong) 91897c95e191d293eb27d8af15cbeafc5b8f3895 guix: Improve guix-build.sh documentation (Carl Dong) 570d769c6c59b9f6d1a2b95b2ed60432cb33b3ba guix: Build support for Windows (Carl Dong) Pull request description: ~~Based on: https://github.com/bitcoin/bitcoin/pull/16519~~ Based on: #17933 (Time Machines are... shall we say... superior :grin:) This PR allows us to perform Guix builds for the `x86_64-w64-mingw32` target. We do this _without_ splitting up the build script like we do in Gitian by using this newfangled alien technology called `case` statements. (This is WIP and might be changed to `if` statements soon) ACKs for top commit: fanquake: ACK a35e3235891d35daa167116cc70340140e883f06 2/3 Tree-SHA512: c471951c23eb2cda919a71285d8b8f2580cb20f09d5db17b53e13dbd8813e01b3e7a83ea848e4913fd0f2bc12c6c133c5f76b54e65c0d89fed4dfd2e0be19875
2020-04-12 12:08:46 +02:00
(define-public glibc-2.24
(package
(inherit glibc)
(version "2.24")
(source (origin
(method git-fetch)
(uri (git-reference
(url "https://sourceware.org/git/glibc.git")
(commit "0d7f1ed30969886c8dde62fbf7d2c79967d4bace")))
(file-name (git-file-name "glibc" "0d7f1ed30969886c8dde62fbf7d2c79967d4bace"))
(sha256
(base32
"0g5hryia5v1k0qx97qffgwzrz4lr4jw3s5kj04yllhswsxyjbic3"))
(patches (search-our-patches "glibc-ldd-x86_64.patch"
"glibc-versioned-locpath.patch"
"glibc-2.24-elfm-loadaddr-dynamic-rewrite.patch"
"glibc-2.24-no-build-time-cxx-header-run.patch"))))))
(define glibc-2.27/bitcoin-patched
(package-with-extra-patches glibc-2.27
(search-our-patches "glibc-2.27-riscv64-Use-__has_include__-to-include-asm-syscalls.h.patch")))
(define-public lief
(package
(name "python-lief")
(version "0.11.5")
(source
(origin
(method git-fetch)
(uri (git-reference
(url "https://github.com/lief-project/LIEF.git")
(commit version)))
(file-name (git-file-name name version))
(sha256
(base32
"0qahjfg1n0x76ps2mbyljvws1l3qhkqvmxqbahps4qgywl2hbdkj"))))
(build-system python-build-system)
(native-inputs
`(("cmake" ,cmake)))
(home-page "https://github.com/lief-project/LIEF")
(synopsis "Library to Instrument Executable Formats")
(description "Python library to to provide a cross platform library which can
parse, modify and abstract ELF, PE and MachO formats.")
(license license:asl2.0)))
Merge #15277: contrib: Enable building in Guix containers 751549b52a9a4cd27389d807ae67f02bbb39cd7f contrib: guix: Additional clarifications re: substitutes (Carl Dong) cd3e947f50db7cfe05c05b368c25742193729a62 contrib: guix: Various improvements. (Carl Dong) 8dff3e48a9e03299468ed3b342642f01f70da9db contrib: guix: Clarify SOURCE_DATE_EPOCH. (Carl Dong) 3e80ec3ea9691c7c89173de922a113e643fe976b contrib: Add deterministic Guix builds. (Carl Dong) Pull request description: ~~**This post is kept updated as this project progresses. Use this [latest update link](https://github.com/bitcoin/bitcoin/pull/15277#issuecomment-497303718) to see what's new.**~~ Please read the `README.md`. ----- ### Guix Introduction This PR enables building bitcoin in Guix containers. [Guix](https://www.gnu.org/software/guix/manual/en/html_node/Features.html) is a transactional package manager much like Nix, but unlike Nix, it has more of a focus on [bootstrappability](https://www.gnu.org/software/guix/manual/en/html_node/Bootstrapping.html) and [reproducibility](https://www.gnu.org/software/guix/blog/tags/reproducible-builds/) which are attractive for security-sensitive projects like bitcoin. ### Guix Build Walkthrough Please read the `README.md`. [Old instructions no. 4](https://github.com/bitcoin/bitcoin/pull/15277#issuecomment-497303718) [Old instructions no. 3](https://github.com/bitcoin/bitcoin/pull/15277#issuecomment-493827011) [Old instructions no. 2](https://github.com/bitcoin/bitcoin/pull/15277#issuecomment-471658439) <details> <summary>Old instructions no. 1</summary> In this PR, we define a Guix [manifest](https://www.gnu.org/software/guix/manual/en/html_node/Invoking-guix-package.html#profile_002dmanifest) in `contrib/guix/manifest.scm`, which declares what packages we want in our environment. We can then invoke ``` guix environment --manifest=contrib/guix/manifest.scm --container --pure --no-grafts --no-substitutes ``` To have Guix: 1. Build an environment containing the packages we defined in our `contrib/guix/manifest.scm` manifest from the Guix bootstrap binaries (see [bootstrappability](https://www.gnu.org/software/guix/manual/en/html_node/Bootstrapping.html) for more details). 2. Start a container with that environment that has no network access, and no access to the host's filesystem except to the `pwd` that it was started in. 3. Drop you into a shell in that container. > Note: if you don't want to wait hours for Guix to build the entire world from scratch, you can eliminate the `--no-substitutes` option to have Guix download from available binary sources. Note that this convenience doesn't necessarily compromise your security, as you can check that a package was built correctly after the fact using `guix build --check <packagename>` Therefore, we can perform a build of bitcoin much like in Gitian by invoking the following: ``` make -C depends -j"$(nproc)" download && \ cat contrib/guix/build.sh | guix environment --manifest=contrib/guix/manifest.scm --container --pure --no-grafts --no-substitutes ``` We don't include `make -C depends -j"$(nproc)" download` inside `contrib/guix/build.sh` because `contrib/guix/build.sh` is run inside the container, which has no network access (which is a good thing). </details> ### Rationale I believe that this represents a substantial improvement for the "supply chain security" of bitcoin because: 1. We no longer have to rely on Ubuntu for our build environment for our releases ([oh the horror](https://github.com/bitcoin/bitcoin/blob/72bd4ab867e3be0d8410403d9641c08288d343e3/contrib/gitian-descriptors/gitian-linux.yml#L10)), because Guix builds everything about the container, we can perform this on almost any Linux distro/system. 2. It is now much easier to determine what trusted binaries are in our supply chain, and even make a nice visualization! (see [bootstrappability](https://www.gnu.org/software/guix/manual/en/html_node/Bootstrapping.html)). 3. There is active effort among Guix folks to minimize the number of trusted binaries even further. OriansJ's [stage0](https://github.com/oriansj/stage0), and janneke's [Mes](https://www.gnu.org/software/mes/) all aim to achieve [reduced binary boostrap](http://joyofsource.com/reduced-binary-seed-bootstrap.html) for Guix. In fact, I believe if OriansJ gets his way, we will end up some day with only a single trusted binary: hex0 (a ~500 byte self-hosting hex assembler). ### Steps to Completion - [x] Successfully build bitcoin inside the Guix environment - [x] Make `check-symbols` pass - [x] Do the above but without nasty hacks - [x] Solve some of the more innocuous hacks - [ ] Make it cross-compile (HELP WANTED HERE) - [x] Linux - [x] x86_64-linux-gnu - [x] i686-linux-gnu - [x] aarch64-linux-gnu - [x] arm-linux-gnueabihf - [x] riscv64-linux-gnu - [ ] OS X - [ ] x86_64-apple-darwin14 - [ ] Windows - [ ] x86_64-w64-mingw32 - [ ] Maybe make importer for depends syntax - [ ] Document build process for future releases - [ ] Extra: Pin the revision of Guix that we build with with Guix [inferiors](https://www.gnu.org/software/guix/manual/en/html_node/Inferiors.html) ### Help Wanted [Old content no. 3](https://github.com/bitcoin/bitcoin/pull/15277#issuecomment-483318210) [Old content no. 2](https://github.com/bitcoin/bitcoin/pull/15277#issuecomment-471658439) <details> <summary>Old content no. 1</summary> As of now, the command described above to perform a build of bitcoin a lot like Gitian works, but fails at the `check-symbols` stage. This is because a few dynamic libraries are linked in that shouldn't be. Here's what `ldd src/bitcoind` looks like when built in a Guix container: ``` linux-vdso.so.1 (0x00007ffcc2d90000) libdl.so.2 => /gnu/store/h90vnqw0nwd0hhm1l5dgxsdrigddfmq4-glibc-2.28/lib/libdl.so.2 (0x00007fb7eda09000) librt.so.1 => /gnu/store/h90vnqw0nwd0hhm1l5dgxsdrigddfmq4-glibc-2.28/lib/librt.so.1 (0x00007fb7ed9ff000) libstdc++.so.6 => /gnu/store/4sqps8dczv3g7rwbdibfz6rf5jlk7w90-gcc-5.5.0-lib/lib/libstdc++.so.6 (0x00007fb7ed87c000) libpthread.so.0 => /gnu/store/h90vnqw0nwd0hhm1l5dgxsdrigddfmq4-glibc-2.28/lib/libpthread.so.0 (0x00007fb7ed85b000) libm.so.6 => /gnu/store/h90vnqw0nwd0hhm1l5dgxsdrigddfmq4-glibc-2.28/lib/libm.so.6 (0x00007fb7ed6da000) libgcc_s.so.1 => /gnu/store/4sqps8dczv3g7rwbdibfz6rf5jlk7w90-gcc-5.5.0-lib/lib/libgcc_s.so.1 (0x00007fb7ed6bf000) libc.so.6 => /gnu/store/h90vnqw0nwd0hhm1l5dgxsdrigddfmq4-glibc-2.28/lib/libc.so.6 (0x00007fb7ed506000) /gnu/store/h90vnqw0nwd0hhm1l5dgxsdrigddfmq4-glibc-2.28/lib/ld-linux-x86-64.so.2 => /usr/lib64/ld-linux-x86-64.so.2 (0x00007fb7ee3a0000) ``` And here's what it looks in one of our releases: ``` linux-vdso.so.1 (0x00007ffff52cd000) libpthread.so.0 => /usr/lib/libpthread.so.0 (0x00007f87726b4000) librt.so.1 => /usr/lib/librt.so.1 (0x00007f87726aa000) libm.so.6 => /usr/lib/libm.so.6 (0x00007f8772525000) libgcc_s.so.1 => /usr/lib/libgcc_s.so.1 (0x00007f877250b000) libc.so.6 => /usr/lib/libc.so.6 (0x00007f8772347000) /lib64/ld-linux-x86-64.so.2 => /usr/lib64/ld-linux-x86-64.so.2 (0x00007f8773392000) ``` ~~I suspect it is because my script does not apply the gitian-input patches [described in the release process](https://github.com/bitcoin/bitcoin/blob/master/doc/release-process.md#fetch-and-create-inputs-first-time-or-when-dependency-versions-change) but there is no description as to how these patches are applied.~~ It might also be something else entirely. Edit: It is something else. It appears that the gitian inputs are only used by [`gitian-win-signer.yml`](https://github.com/bitcoin/bitcoin/blob/d6e700e40f861ddd6743f4d13f0d6f6bc19093c2/contrib/gitian-descriptors/gitian-win-signer.yml#L14) </details> ### How to Help 1. Install Guix on your distro either [from source](https://www.gnu.org/software/guix/manual/en/html_node/Requirements.html) or perform a [binary installation](https://www.gnu.org/software/guix/manual/en/html_node/Binary-Installation.html#Binary-Installation) 2. Try out my branch and the command described above! ACKs for top commit: MarcoFalke: Thanks for the replies. ACK 751549b52a9a4cd27389d807ae67f02bbb39cd7f laanwj: ACK 751549b52a9a4cd27389d807ae67f02bbb39cd7f Tree-SHA512: 50e6ab58c6bda9a67125b6271daf7eff0ca57d0efa8941ed3cd951e5bf78b31552fc5e537b1e1bcf2d3cc918c63adf19d685aa117a0f851024dc67e697890a8d
2019-07-12 19:23:30 +02:00
(packages->manifest
Merge #17595: guix: Enable building for `x86_64-w64-mingw32` target a35e3235891d35daa167116cc70340140e883f06 guix: Appease travis. (Carl Dong) 0b66d22da5f53640e22f05adf880782c613e6d0f guix: Use gcc-9 for mingw-w64 instead of 8 (Carl Dong) ba0b99bdd613ba7f17c6247ece3001e1b44759a3 guix: Don't set MINGW_HAS_SECURE_API CFLAG in depends (Carl Dong) 93439a71eda49fb69f1e82966a23a946733aa6fa guix: Bump to upstream commit with mingw-w64 changes (Carl Dong) 35a96792dda9e78165b1598aeac7b2ab759e7be5 guix: Check mingw symbols, improve SSP fix docs (Carl Dong) 449d8fe25bbe25daacfc67aa89ca32b0a3254c5a guix: Expand on INT trap message (Carl Dong) 3f1f03c67a8e9edf487f08d272adb18b0a3942c8 guix: Spelling fixes (Carl Dong) ff821dd2a1c600488d11e7d9a20e9179ecc9144b guix: Reinstate make-ssp-fixed-gcc (Carl Dong) 360a9e0ad50a36ec79a1a160dbed3966689fd41c guix: Bump time-machine for mingw-w64 patches (Carl Dong) 93e41b7e3b54c17fd1b4c61ee95fc0dc2827e954 guix: Use gcc-8 for mingw-w64 instead of 7 (Carl Dong) ef4f7e4c45c60a69406134122f091c77c6ef740f guix: Set the well-known timezone env var (Carl Dong) acf4b3b3b5accf60a19441a0298ef27001b78e72 guix: Make x86_64-w64-mingw32 builds reproducible (Carl Dong) c4cce00eac691625b78b92f7dba0b7f57def19e5 guix: Remove dead links from README. (Carl Dong) df953a4c9a6143f45864757b706c88b6fa70545a guix: Appease shellcheck. (Carl Dong) 91897c95e191d293eb27d8af15cbeafc5b8f3895 guix: Improve guix-build.sh documentation (Carl Dong) 570d769c6c59b9f6d1a2b95b2ed60432cb33b3ba guix: Build support for Windows (Carl Dong) Pull request description: ~~Based on: https://github.com/bitcoin/bitcoin/pull/16519~~ Based on: #17933 (Time Machines are... shall we say... superior :grin:) This PR allows us to perform Guix builds for the `x86_64-w64-mingw32` target. We do this _without_ splitting up the build script like we do in Gitian by using this newfangled alien technology called `case` statements. (This is WIP and might be changed to `if` statements soon) ACKs for top commit: fanquake: ACK a35e3235891d35daa167116cc70340140e883f06 2/3 Tree-SHA512: c471951c23eb2cda919a71285d8b8f2580cb20f09d5db17b53e13dbd8813e01b3e7a83ea848e4913fd0f2bc12c6c133c5f76b54e65c0d89fed4dfd2e0be19875
2020-04-12 12:08:46 +02:00
(append
(list ;; The Basics
bash
Merge #17595: guix: Enable building for `x86_64-w64-mingw32` target a35e3235891d35daa167116cc70340140e883f06 guix: Appease travis. (Carl Dong) 0b66d22da5f53640e22f05adf880782c613e6d0f guix: Use gcc-9 for mingw-w64 instead of 8 (Carl Dong) ba0b99bdd613ba7f17c6247ece3001e1b44759a3 guix: Don't set MINGW_HAS_SECURE_API CFLAG in depends (Carl Dong) 93439a71eda49fb69f1e82966a23a946733aa6fa guix: Bump to upstream commit with mingw-w64 changes (Carl Dong) 35a96792dda9e78165b1598aeac7b2ab759e7be5 guix: Check mingw symbols, improve SSP fix docs (Carl Dong) 449d8fe25bbe25daacfc67aa89ca32b0a3254c5a guix: Expand on INT trap message (Carl Dong) 3f1f03c67a8e9edf487f08d272adb18b0a3942c8 guix: Spelling fixes (Carl Dong) ff821dd2a1c600488d11e7d9a20e9179ecc9144b guix: Reinstate make-ssp-fixed-gcc (Carl Dong) 360a9e0ad50a36ec79a1a160dbed3966689fd41c guix: Bump time-machine for mingw-w64 patches (Carl Dong) 93e41b7e3b54c17fd1b4c61ee95fc0dc2827e954 guix: Use gcc-8 for mingw-w64 instead of 7 (Carl Dong) ef4f7e4c45c60a69406134122f091c77c6ef740f guix: Set the well-known timezone env var (Carl Dong) acf4b3b3b5accf60a19441a0298ef27001b78e72 guix: Make x86_64-w64-mingw32 builds reproducible (Carl Dong) c4cce00eac691625b78b92f7dba0b7f57def19e5 guix: Remove dead links from README. (Carl Dong) df953a4c9a6143f45864757b706c88b6fa70545a guix: Appease shellcheck. (Carl Dong) 91897c95e191d293eb27d8af15cbeafc5b8f3895 guix: Improve guix-build.sh documentation (Carl Dong) 570d769c6c59b9f6d1a2b95b2ed60432cb33b3ba guix: Build support for Windows (Carl Dong) Pull request description: ~~Based on: https://github.com/bitcoin/bitcoin/pull/16519~~ Based on: #17933 (Time Machines are... shall we say... superior :grin:) This PR allows us to perform Guix builds for the `x86_64-w64-mingw32` target. We do this _without_ splitting up the build script like we do in Gitian by using this newfangled alien technology called `case` statements. (This is WIP and might be changed to `if` statements soon) ACKs for top commit: fanquake: ACK a35e3235891d35daa167116cc70340140e883f06 2/3 Tree-SHA512: c471951c23eb2cda919a71285d8b8f2580cb20f09d5db17b53e13dbd8813e01b3e7a83ea848e4913fd0f2bc12c6c133c5f76b54e65c0d89fed4dfd2e0be19875
2020-04-12 12:08:46 +02:00
which
coreutils
util-linux
;; File(system) inspection
file
grep
diffutils
findutils
;; File transformation
patch
gawk
sed
Merge bitcoin/bitcoin#22182: guix: Overhaul how guix-{attest,verify} works and hierarchy e2c40a4ed5272d72fea997bd936fba28bb753226 guix-attest: Error out if SHA256SUMS is unexpected (Carl Dong) 4cc35daed557f38b080360a89036b2e97a6f78c2 Rewrite guix-{attest,verify} for new hier (Carl Dong) 28a9c9b83924f585b397f0f3b8e9e73780ac0ad6 Make SHA256SUMS fragment right after build (Carl Dong) Pull request description: Based on: #22075 Code reviewers: I recommend reading the new `guix-{attest,verify}` files instead of trying to read the diff The following changes resolve many usability improvements which were pointed out to me: 1. Some maintainers like to extract their "uncodesigned tarball" inside the `output/` directory, resulting in the older `guix-attest` mistakenly attesting to the extracted contents 2. Maintainers whose GPG keys reside on an external smartcard often need to physically interact with the smartcard as a way to approve the signing operation, having one signature per platform means a lot of fidgeting 3. Maintainers wishing to sign on a separate machine now has the option of transferring only a subtree of `output/`, namely `output/*/SHA256SUMS.part`, in order to perform a signature (you may need to specify an `$OUTDIR_BASE` env var) 4. An `all.SHA256SUMS` file should be usable as the base `SHA256SUMS` in bitcoin core torrents and on the release server. For those who sign on an separate machine than the one you do builds on, the following steps will work: 1. `env GUIX_SIGS_REPO=/home/achow101/guix.sigs SIGNER=achow101 NO_SIGN=1 ./contrib/guix/guix-attest` 2. Copy `/home/achow101/guix.sigs/<tag>/achow101` (which does not yet have signatures) to signing machine 3. Sign the `SHA256SUMS` files: ```bash for i in "<path-to-achow101>/*.SHA256SUMS"; do gpg --detach-sign --local-user "<your-key-here>" --armor --output "$i"{.asc,} done ``` 5. Upload `<path-to-achow101>` (now with signatures) to `guix.sigs` ----- After this change, output directories will now include a `SHA256SUMS.part` fragment, created immediately after a successful build: ``` output └── x86_64-w64-mingw32 ├── bitcoin-4e069f7589da-win64-debug.zip ├── bitcoin-4e069f7589da-win64-setup-unsigned.exe ├── bitcoin-4e069f7589da-win64.zip ├── bitcoin-4e069f7589da-win-unsigned.tar.gz └── SHA256SUMS.part ``` These `SHA256SUMS.part` fragments look something like: ``` 3ebd7262b1a0a5bb757fef1f70e7e14033c70f98c059bc4dbfee5d1992b25825 dist-archive/bitcoin-4e069f7589da.tar.gz def2e7d3de5ab3e3f955344e75151df4f33713f9101f5295bd13c9375bdf633b x86_64-w64-mingw32/bitcoin-4e069f7589da-win64-debug.zip 643049fe3ee4a4e83a1739607e67b11b7c9b1a66208a6f35a9ff634ba795500e x86_64-w64-mingw32/bitcoin-4e069f7589da-win64-setup-unsigned.exe a247a1ccec0ccc2e138c648284bd01f6a761f2d8d6d07d91b5b4a6670ec3f288 x86_64-w64-mingw32/bitcoin-4e069f7589da-win-unsigned.tar.gz fab76a836dcc592e39c04fd2396696633fb6eb56e39ecbf6c909bd173ed4280c x86_64-w64-mingw32/bitcoin-4e069f7589da-win64.zip ``` Meaning that they are valid `SHA256SUMS` files when `sha256sum --check`'d at the `guix-build-*/output` directory level When `guix-attest` is invoked, these `SHA256SUMS.part` files are combined and sorted (by `-k2`, `LC_ALL=C`) to create: 1. `noncodesigned.SHA256SUMS` for a manifest of all non-codesigned outputs, and 3. `all.SHA256SUMS` for a manifest of all outputs including non-codesigned outputs Then both files are signed, resulting in the following `guix.sigs` hierarchy: ``` 4e069f7589da/ └── dongcarl ├── all.SHA256SUMS ├── all.SHA256SUMS.asc ├── noncodesigned.SHA256SUMS └── noncodesigned.SHA256SUMS.asc ``` ACKs for top commit: achow101: ACK e2c40a4ed5272d72fea997bd936fba28bb753226 hebasto: ACK e2c40a4ed5272d72fea997bd936fba28bb753226, tested on Linux Mint 20.1 (x86_64) with and w/o `NO_SIGN=1`. Changes in `contrib/guix/libexec/codesign.sh` and `contrib/guix/guix-verify` are reviewed only. Tree-SHA512: 618aacefb0eb6595735a9ab6a98ea6598fce65f9ccf33fa1e7ef93bf140c0f6cfc16e34870c6aa3e4777dd3f004b92a82a994141879870141742df948ec59c1f
2021-06-17 07:10:29 +02:00
moreutils
Merge #17595: guix: Enable building for `x86_64-w64-mingw32` target a35e3235891d35daa167116cc70340140e883f06 guix: Appease travis. (Carl Dong) 0b66d22da5f53640e22f05adf880782c613e6d0f guix: Use gcc-9 for mingw-w64 instead of 8 (Carl Dong) ba0b99bdd613ba7f17c6247ece3001e1b44759a3 guix: Don't set MINGW_HAS_SECURE_API CFLAG in depends (Carl Dong) 93439a71eda49fb69f1e82966a23a946733aa6fa guix: Bump to upstream commit with mingw-w64 changes (Carl Dong) 35a96792dda9e78165b1598aeac7b2ab759e7be5 guix: Check mingw symbols, improve SSP fix docs (Carl Dong) 449d8fe25bbe25daacfc67aa89ca32b0a3254c5a guix: Expand on INT trap message (Carl Dong) 3f1f03c67a8e9edf487f08d272adb18b0a3942c8 guix: Spelling fixes (Carl Dong) ff821dd2a1c600488d11e7d9a20e9179ecc9144b guix: Reinstate make-ssp-fixed-gcc (Carl Dong) 360a9e0ad50a36ec79a1a160dbed3966689fd41c guix: Bump time-machine for mingw-w64 patches (Carl Dong) 93e41b7e3b54c17fd1b4c61ee95fc0dc2827e954 guix: Use gcc-8 for mingw-w64 instead of 7 (Carl Dong) ef4f7e4c45c60a69406134122f091c77c6ef740f guix: Set the well-known timezone env var (Carl Dong) acf4b3b3b5accf60a19441a0298ef27001b78e72 guix: Make x86_64-w64-mingw32 builds reproducible (Carl Dong) c4cce00eac691625b78b92f7dba0b7f57def19e5 guix: Remove dead links from README. (Carl Dong) df953a4c9a6143f45864757b706c88b6fa70545a guix: Appease shellcheck. (Carl Dong) 91897c95e191d293eb27d8af15cbeafc5b8f3895 guix: Improve guix-build.sh documentation (Carl Dong) 570d769c6c59b9f6d1a2b95b2ed60432cb33b3ba guix: Build support for Windows (Carl Dong) Pull request description: ~~Based on: https://github.com/bitcoin/bitcoin/pull/16519~~ Based on: #17933 (Time Machines are... shall we say... superior :grin:) This PR allows us to perform Guix builds for the `x86_64-w64-mingw32` target. We do this _without_ splitting up the build script like we do in Gitian by using this newfangled alien technology called `case` statements. (This is WIP and might be changed to `if` statements soon) ACKs for top commit: fanquake: ACK a35e3235891d35daa167116cc70340140e883f06 2/3 Tree-SHA512: c471951c23eb2cda919a71285d8b8f2580cb20f09d5db17b53e13dbd8813e01b3e7a83ea848e4913fd0f2bc12c6c133c5f76b54e65c0d89fed4dfd2e0be19875
2020-04-12 12:08:46 +02:00
;; Compression and archiving
tar
bzip2
gzip
xz
zlib
Merge #21375: guix: Misc feedback-based fixes + hier restructuring 7476b46f1893a4858616d2a8456a7c43238851ed guix: Build dmg as a static binary (Carl Dong) 06d6cf6784421290e6235fe8684d5e08ed6f1b62 depends: libdmg-hfsplus: Skip CMake RPATH patching (Carl Dong) 65176ab5730dff34466caaecdd292625ef8294fc guix: Remove codesign_allocate+pagestuff from unsigned tarball (Carl Dong) ca85679eb43b8375a95d82101977829d08fb1e1b guix: Use clang-toolchain instead of clang (Carl Dong) 1aec0eda8fd31a57b0621eea616398017c2ead98 guix: Fallback to local build for substitute-enabled Guix users (Carl Dong) 1742f8e12d163852df09575e03edcd3db73198ee guix: Add early health check for guix-daemon (Carl Dong) c1ae726a13ecfa5e7e9fdc3030a8110b8bb263f8 guix: More thoroughly control native toolchain (Carl Dong) 39741128d3775d198dbee34dc827353bfd18acd8 guix: Supply --link-profile (Carl Dong) d55a1056ee565afed64e42d6f6efb6b0adc5599b guix: Add troubleshooting documentation entries (Carl Dong) 7f401c953f8bb3574cec48561e13ef3b47dedc6e guix: Adapt guix-build to prelude, restructure hier (Carl Dong) 4eccf063b252bfe256cf72d363a24cf0183e926e guix: Remove guix-build.sh filename extension (Carl Dong) 7753357a7bae98ec775c707b9dec4cea1e945802 guix: Add source-able bash prelude and utils (Carl Dong) e5b49a01f5d0f631e7f08f86ca8a2c2b8213319f guix: Create windeploy inside distsrc-* (Carl Dong) 3e9982ab3877eb8fe0a8c0cb3d847ac0913c7336 contrib: Silence git-describe when looking for tag (Carl Dong) d5a71e97853ea9e1b879e8c76bfb01d4bef33172 guix: Use --cores instead of --max-jobs (Carl Dong) Pull request description: This PR addresses a few hiccups encountered by the brave souls who've been experimenting with the Guix scripts: - Resolves confusion between `--cores=` and `--max-jobs=` - `guix`'s `--cores=` actually corresponds to make's `--jobs=`, so let's just control `--cores=` with our overridable env var - `git-describe` will scream `fatal: no tag exactly matches '<hash>'` when looking for a tag, but we don't care, so silence that - `windeploy/unsigned` should be inside `distsrc-*` and created idempotently (sorry I know this one annoyed people) - Add troubleshooting documentation to `README.md` - Add early health check for `guix-daemon` in case user forgot to start a `guix-daemon` - Depending on configuration, a `--fallback` flag may be needed to tell Guix to not fail if substitutes fail but fallback to building locally - `codesign_allocate` and `pagestuff` are now unnecessary for codesigning as we're now using `signapple` A few robustness changes are also included: - We supply the `--link-profile` flag, as some Guix packages may expect the profile to be available under `$HOME/.guix-profile` - We now clear and manually set all toolchain-related env vars (e.g. `C*_INCLUDE_PATH`) ourselves, after patching a Qt::moc bug - We use the native `clang-toolchain` package for darwin builds instead of `clang`, lining up with all our other toolchain packages. Finally, we restructure the guix building hierarchy such that it looks something like: ``` guix-build-<short-hash-or-version-tag> ├── distsrc-<short-hash-or-version-tag>-${HOST} │ ├── contrib │ ├── depends │ ├── src │ └── ... ├── distsrc-<short-hash-or-version-tag>-... └── output ├── dist-archive │ └── bitcoin-<short-hash-or-version-tag>.tar.gz ├── *-linux-* │ ├── bitcoin-<short-hash-or-version-tag>-*-linux-*-debug.tar.gz │ └── bitcoin-<short-hash-or-version-tag>-*-linux-*.tar.gz ├── x86_64-apple-darwin18 │ ├── bitcoin-<short-hash-or-version-tag>-osx64.tar.gz │ ├── bitcoin-<short-hash-or-version-tag>-osx-unsigned.dmg │ └── bitcoin-<short-hash-or-version-tag>-osx-unsigned.tar.gz └── x86_64-w64-mingw32 ├── bitcoin-<short-hash-or-version-tag>-win64-debug.zip ├── bitcoin-<short-hash-or-version-tag>-win64-setup-unsigned.exe ├── bitcoin-<short-hash-or-version-tag>-win64.zip └── bitcoin-<short-hash-or-version-tag>-win-unsigned.tar.gz ``` Separating guix builds by their version identifier (basically namespacing them) allows us to change the layout in the future without worry about potential naming conflicts. ACKs for top commit: sipa: ACK 7476b46f1893a4858616d2a8456a7c43238851ed laanwj: ACK 7476b46f1893a4858616d2a8456a7c43238851ed Tree-SHA512: 0e899aa941aafdf552b2a7e8a08131ee9283180bbef7334439e2461a02aa7235ab7b9ca9c149b80fc5d0a9f4bbd35bc80fcee26197c0836ba8eaf2d86ffa0386
2021-04-06 00:59:11 +02:00
(list zlib "static")
Merge #17595: guix: Enable building for `x86_64-w64-mingw32` target a35e3235891d35daa167116cc70340140e883f06 guix: Appease travis. (Carl Dong) 0b66d22da5f53640e22f05adf880782c613e6d0f guix: Use gcc-9 for mingw-w64 instead of 8 (Carl Dong) ba0b99bdd613ba7f17c6247ece3001e1b44759a3 guix: Don't set MINGW_HAS_SECURE_API CFLAG in depends (Carl Dong) 93439a71eda49fb69f1e82966a23a946733aa6fa guix: Bump to upstream commit with mingw-w64 changes (Carl Dong) 35a96792dda9e78165b1598aeac7b2ab759e7be5 guix: Check mingw symbols, improve SSP fix docs (Carl Dong) 449d8fe25bbe25daacfc67aa89ca32b0a3254c5a guix: Expand on INT trap message (Carl Dong) 3f1f03c67a8e9edf487f08d272adb18b0a3942c8 guix: Spelling fixes (Carl Dong) ff821dd2a1c600488d11e7d9a20e9179ecc9144b guix: Reinstate make-ssp-fixed-gcc (Carl Dong) 360a9e0ad50a36ec79a1a160dbed3966689fd41c guix: Bump time-machine for mingw-w64 patches (Carl Dong) 93e41b7e3b54c17fd1b4c61ee95fc0dc2827e954 guix: Use gcc-8 for mingw-w64 instead of 7 (Carl Dong) ef4f7e4c45c60a69406134122f091c77c6ef740f guix: Set the well-known timezone env var (Carl Dong) acf4b3b3b5accf60a19441a0298ef27001b78e72 guix: Make x86_64-w64-mingw32 builds reproducible (Carl Dong) c4cce00eac691625b78b92f7dba0b7f57def19e5 guix: Remove dead links from README. (Carl Dong) df953a4c9a6143f45864757b706c88b6fa70545a guix: Appease shellcheck. (Carl Dong) 91897c95e191d293eb27d8af15cbeafc5b8f3895 guix: Improve guix-build.sh documentation (Carl Dong) 570d769c6c59b9f6d1a2b95b2ed60432cb33b3ba guix: Build support for Windows (Carl Dong) Pull request description: ~~Based on: https://github.com/bitcoin/bitcoin/pull/16519~~ Based on: #17933 (Time Machines are... shall we say... superior :grin:) This PR allows us to perform Guix builds for the `x86_64-w64-mingw32` target. We do this _without_ splitting up the build script like we do in Gitian by using this newfangled alien technology called `case` statements. (This is WIP and might be changed to `if` statements soon) ACKs for top commit: fanquake: ACK a35e3235891d35daa167116cc70340140e883f06 2/3 Tree-SHA512: c471951c23eb2cda919a71285d8b8f2580cb20f09d5db17b53e13dbd8813e01b3e7a83ea848e4913fd0f2bc12c6c133c5f76b54e65c0d89fed4dfd2e0be19875
2020-04-12 12:08:46 +02:00
;; Build tools
gnu-make
libtool
autoconf
automake
pkg-config
bison
;; Scripting
perl
python-3
;; Git
git
;; Tests
lief
;; Native gcc 7 toolchain
Merge #21375: guix: Misc feedback-based fixes + hier restructuring 7476b46f1893a4858616d2a8456a7c43238851ed guix: Build dmg as a static binary (Carl Dong) 06d6cf6784421290e6235fe8684d5e08ed6f1b62 depends: libdmg-hfsplus: Skip CMake RPATH patching (Carl Dong) 65176ab5730dff34466caaecdd292625ef8294fc guix: Remove codesign_allocate+pagestuff from unsigned tarball (Carl Dong) ca85679eb43b8375a95d82101977829d08fb1e1b guix: Use clang-toolchain instead of clang (Carl Dong) 1aec0eda8fd31a57b0621eea616398017c2ead98 guix: Fallback to local build for substitute-enabled Guix users (Carl Dong) 1742f8e12d163852df09575e03edcd3db73198ee guix: Add early health check for guix-daemon (Carl Dong) c1ae726a13ecfa5e7e9fdc3030a8110b8bb263f8 guix: More thoroughly control native toolchain (Carl Dong) 39741128d3775d198dbee34dc827353bfd18acd8 guix: Supply --link-profile (Carl Dong) d55a1056ee565afed64e42d6f6efb6b0adc5599b guix: Add troubleshooting documentation entries (Carl Dong) 7f401c953f8bb3574cec48561e13ef3b47dedc6e guix: Adapt guix-build to prelude, restructure hier (Carl Dong) 4eccf063b252bfe256cf72d363a24cf0183e926e guix: Remove guix-build.sh filename extension (Carl Dong) 7753357a7bae98ec775c707b9dec4cea1e945802 guix: Add source-able bash prelude and utils (Carl Dong) e5b49a01f5d0f631e7f08f86ca8a2c2b8213319f guix: Create windeploy inside distsrc-* (Carl Dong) 3e9982ab3877eb8fe0a8c0cb3d847ac0913c7336 contrib: Silence git-describe when looking for tag (Carl Dong) d5a71e97853ea9e1b879e8c76bfb01d4bef33172 guix: Use --cores instead of --max-jobs (Carl Dong) Pull request description: This PR addresses a few hiccups encountered by the brave souls who've been experimenting with the Guix scripts: - Resolves confusion between `--cores=` and `--max-jobs=` - `guix`'s `--cores=` actually corresponds to make's `--jobs=`, so let's just control `--cores=` with our overridable env var - `git-describe` will scream `fatal: no tag exactly matches '<hash>'` when looking for a tag, but we don't care, so silence that - `windeploy/unsigned` should be inside `distsrc-*` and created idempotently (sorry I know this one annoyed people) - Add troubleshooting documentation to `README.md` - Add early health check for `guix-daemon` in case user forgot to start a `guix-daemon` - Depending on configuration, a `--fallback` flag may be needed to tell Guix to not fail if substitutes fail but fallback to building locally - `codesign_allocate` and `pagestuff` are now unnecessary for codesigning as we're now using `signapple` A few robustness changes are also included: - We supply the `--link-profile` flag, as some Guix packages may expect the profile to be available under `$HOME/.guix-profile` - We now clear and manually set all toolchain-related env vars (e.g. `C*_INCLUDE_PATH`) ourselves, after patching a Qt::moc bug - We use the native `clang-toolchain` package for darwin builds instead of `clang`, lining up with all our other toolchain packages. Finally, we restructure the guix building hierarchy such that it looks something like: ``` guix-build-<short-hash-or-version-tag> ├── distsrc-<short-hash-or-version-tag>-${HOST} │ ├── contrib │ ├── depends │ ├── src │ └── ... ├── distsrc-<short-hash-or-version-tag>-... └── output ├── dist-archive │ └── bitcoin-<short-hash-or-version-tag>.tar.gz ├── *-linux-* │ ├── bitcoin-<short-hash-or-version-tag>-*-linux-*-debug.tar.gz │ └── bitcoin-<short-hash-or-version-tag>-*-linux-*.tar.gz ├── x86_64-apple-darwin18 │ ├── bitcoin-<short-hash-or-version-tag>-osx64.tar.gz │ ├── bitcoin-<short-hash-or-version-tag>-osx-unsigned.dmg │ └── bitcoin-<short-hash-or-version-tag>-osx-unsigned.tar.gz └── x86_64-w64-mingw32 ├── bitcoin-<short-hash-or-version-tag>-win64-debug.zip ├── bitcoin-<short-hash-or-version-tag>-win64-setup-unsigned.exe ├── bitcoin-<short-hash-or-version-tag>-win64.zip └── bitcoin-<short-hash-or-version-tag>-win-unsigned.tar.gz ``` Separating guix builds by their version identifier (basically namespacing them) allows us to change the layout in the future without worry about potential naming conflicts. ACKs for top commit: sipa: ACK 7476b46f1893a4858616d2a8456a7c43238851ed laanwj: ACK 7476b46f1893a4858616d2a8456a7c43238851ed Tree-SHA512: 0e899aa941aafdf552b2a7e8a08131ee9283180bbef7334439e2461a02aa7235ab7b9ca9c149b80fc5d0a9f4bbd35bc80fcee26197c0836ba8eaf2d86ffa0386
2021-04-06 00:59:11 +02:00
gcc-toolchain-7
(list gcc-toolchain-7 "static"))
Merge #17595: guix: Enable building for `x86_64-w64-mingw32` target a35e3235891d35daa167116cc70340140e883f06 guix: Appease travis. (Carl Dong) 0b66d22da5f53640e22f05adf880782c613e6d0f guix: Use gcc-9 for mingw-w64 instead of 8 (Carl Dong) ba0b99bdd613ba7f17c6247ece3001e1b44759a3 guix: Don't set MINGW_HAS_SECURE_API CFLAG in depends (Carl Dong) 93439a71eda49fb69f1e82966a23a946733aa6fa guix: Bump to upstream commit with mingw-w64 changes (Carl Dong) 35a96792dda9e78165b1598aeac7b2ab759e7be5 guix: Check mingw symbols, improve SSP fix docs (Carl Dong) 449d8fe25bbe25daacfc67aa89ca32b0a3254c5a guix: Expand on INT trap message (Carl Dong) 3f1f03c67a8e9edf487f08d272adb18b0a3942c8 guix: Spelling fixes (Carl Dong) ff821dd2a1c600488d11e7d9a20e9179ecc9144b guix: Reinstate make-ssp-fixed-gcc (Carl Dong) 360a9e0ad50a36ec79a1a160dbed3966689fd41c guix: Bump time-machine for mingw-w64 patches (Carl Dong) 93e41b7e3b54c17fd1b4c61ee95fc0dc2827e954 guix: Use gcc-8 for mingw-w64 instead of 7 (Carl Dong) ef4f7e4c45c60a69406134122f091c77c6ef740f guix: Set the well-known timezone env var (Carl Dong) acf4b3b3b5accf60a19441a0298ef27001b78e72 guix: Make x86_64-w64-mingw32 builds reproducible (Carl Dong) c4cce00eac691625b78b92f7dba0b7f57def19e5 guix: Remove dead links from README. (Carl Dong) df953a4c9a6143f45864757b706c88b6fa70545a guix: Appease shellcheck. (Carl Dong) 91897c95e191d293eb27d8af15cbeafc5b8f3895 guix: Improve guix-build.sh documentation (Carl Dong) 570d769c6c59b9f6d1a2b95b2ed60432cb33b3ba guix: Build support for Windows (Carl Dong) Pull request description: ~~Based on: https://github.com/bitcoin/bitcoin/pull/16519~~ Based on: #17933 (Time Machines are... shall we say... superior :grin:) This PR allows us to perform Guix builds for the `x86_64-w64-mingw32` target. We do this _without_ splitting up the build script like we do in Gitian by using this newfangled alien technology called `case` statements. (This is WIP and might be changed to `if` statements soon) ACKs for top commit: fanquake: ACK a35e3235891d35daa167116cc70340140e883f06 2/3 Tree-SHA512: c471951c23eb2cda919a71285d8b8f2580cb20f09d5db17b53e13dbd8813e01b3e7a83ea848e4913fd0f2bc12c6c133c5f76b54e65c0d89fed4dfd2e0be19875
2020-04-12 12:08:46 +02:00
(let ((target (getenv "HOST")))
(cond ((string-suffix? "-mingw32" target)
;; Windows
(list zip
(make-mingw-pthreads-cross-toolchain "x86_64-w64-mingw32")
Merge bitcoin/bitcoin#21239: guix: Add codesignature attachment support for osx+win ee883201cf134952284632e9e9ae72bf1c8c792f guix: repro: Sort find output in libtool for gcc-8 (Carl Dong) ee0a67c32a8861eab650bf8894af06807578eba0 codesigning: Use SHA256 as digest for osslsigncode (Windows) (Carl Dong) 38eb91eb0616ed6dbe34c23e11588d130fef07f8 guix: Add codesigning functionality (Carl Dong) bac2690e6f683fcedb883fe1d32f3c33c628a141 guix: Package codesigning tools (Carl Dong) 0a2176d47767972e4cd5ed302c1dbeedece1708b guix: Reindent existing manifest.scm (Carl Dong) c090a3e9238ba2df07875b4708e908d8dca4ed9b Makefile.am: use APP_DIST_DIR instead of hard-coding dist (Carl Dong) Pull request description: This is the last PR before we reach feature-parity with the Gitian process! Note: I tried using the `Makefile` inside the distsrc to make the dmg instead of manually listing out the commands, but `make` seems to want to re-make a lot of other files which broke the dmg. The workflow looks something like this: 1. `env [ FOO=bar... ] ./contrib/guix/guix-build` (add additional env vars as necessary) 2. Codesigners only: 1. Copy `guix-build-<short-id>/output/x86_64-apple-darwin18/bitcoin-<short-id>-osx-unsigned.tar.gz` and `guix-build-<short-id>/output/x86_64-w64-mingw32/bitcoin-<short-id>-win-unsigned.tar.gz` to signing computer 2. Codesign with `./detached-sig-create.sh` inside the tarball 3. Upload contents of `signature-{osx,win}.tar.gz` to https://github.com/bitcoin-core/bitcoin-detached-sigs (as a new tag) 3. Checkout new tag for `bitcoin-core/bitcoin-detached-sigs` with the detached signatures 4. `env [ FOO=bar... ] DETACHED_SIGS_REPO=<path/to/bitcoin-detached-sigs> ./contrib/guix/guix-codesign` (modify env vars as necessary) 5. Make sure `guix.sigs` is cloned and updated 6. `env GUIX_SIGS_REPO=<path/to/guix.sigs> SIGNER=0x96AB007F1A7ED999=dongcarl ./contrib/guix/guix-attest` (modify env vars as necessary) 7. Commit your new signatures and SHA256SUMS in `guix.sigs` 8. Optionally, after there are multiple signatures in `guix.sigs`: `env GUIX_SIGS_REPO=<path/to/guix.sigs> ./contrib/guix/guix-verify` ACKs for top commit: laanwj: Tested ACK ee883201cf134952284632e9e9ae72bf1c8c792f achow101: ACK ee883201cf134952284632e9e9ae72bf1c8c792f Tree-SHA512: e812a07a5f19f900600c70cb9c717769ef544a6c0c12760b5558b76b6b37df863257f3dbf38b0757e6e06e334470267e94c9f2bdbc27409d6837b1a0bfc6acbc
2021-05-24 15:31:05 +02:00
(make-nsis-with-sde-support nsis-x86_64)
osslsigncode))
Merge #17595: guix: Enable building for `x86_64-w64-mingw32` target a35e3235891d35daa167116cc70340140e883f06 guix: Appease travis. (Carl Dong) 0b66d22da5f53640e22f05adf880782c613e6d0f guix: Use gcc-9 for mingw-w64 instead of 8 (Carl Dong) ba0b99bdd613ba7f17c6247ece3001e1b44759a3 guix: Don't set MINGW_HAS_SECURE_API CFLAG in depends (Carl Dong) 93439a71eda49fb69f1e82966a23a946733aa6fa guix: Bump to upstream commit with mingw-w64 changes (Carl Dong) 35a96792dda9e78165b1598aeac7b2ab759e7be5 guix: Check mingw symbols, improve SSP fix docs (Carl Dong) 449d8fe25bbe25daacfc67aa89ca32b0a3254c5a guix: Expand on INT trap message (Carl Dong) 3f1f03c67a8e9edf487f08d272adb18b0a3942c8 guix: Spelling fixes (Carl Dong) ff821dd2a1c600488d11e7d9a20e9179ecc9144b guix: Reinstate make-ssp-fixed-gcc (Carl Dong) 360a9e0ad50a36ec79a1a160dbed3966689fd41c guix: Bump time-machine for mingw-w64 patches (Carl Dong) 93e41b7e3b54c17fd1b4c61ee95fc0dc2827e954 guix: Use gcc-8 for mingw-w64 instead of 7 (Carl Dong) ef4f7e4c45c60a69406134122f091c77c6ef740f guix: Set the well-known timezone env var (Carl Dong) acf4b3b3b5accf60a19441a0298ef27001b78e72 guix: Make x86_64-w64-mingw32 builds reproducible (Carl Dong) c4cce00eac691625b78b92f7dba0b7f57def19e5 guix: Remove dead links from README. (Carl Dong) df953a4c9a6143f45864757b706c88b6fa70545a guix: Appease shellcheck. (Carl Dong) 91897c95e191d293eb27d8af15cbeafc5b8f3895 guix: Improve guix-build.sh documentation (Carl Dong) 570d769c6c59b9f6d1a2b95b2ed60432cb33b3ba guix: Build support for Windows (Carl Dong) Pull request description: ~~Based on: https://github.com/bitcoin/bitcoin/pull/16519~~ Based on: #17933 (Time Machines are... shall we say... superior :grin:) This PR allows us to perform Guix builds for the `x86_64-w64-mingw32` target. We do this _without_ splitting up the build script like we do in Gitian by using this newfangled alien technology called `case` statements. (This is WIP and might be changed to `if` statements soon) ACKs for top commit: fanquake: ACK a35e3235891d35daa167116cc70340140e883f06 2/3 Tree-SHA512: c471951c23eb2cda919a71285d8b8f2580cb20f09d5db17b53e13dbd8813e01b3e7a83ea848e4913fd0f2bc12c6c133c5f76b54e65c0d89fed4dfd2e0be19875
2020-04-12 12:08:46 +02:00
((string-contains target "-linux-")
(list (cond ((string-contains target "riscv64-")
(make-bitcoin-cross-toolchain target
#:base-libc glibc-2.27/bitcoin-patched
#:base-kernel-headers linux-libre-headers-4.19))
(else
(make-bitcoin-cross-toolchain target)))))
((string-contains target "darwin")
Merge bitcoin/bitcoin#21239: guix: Add codesignature attachment support for osx+win ee883201cf134952284632e9e9ae72bf1c8c792f guix: repro: Sort find output in libtool for gcc-8 (Carl Dong) ee0a67c32a8861eab650bf8894af06807578eba0 codesigning: Use SHA256 as digest for osslsigncode (Windows) (Carl Dong) 38eb91eb0616ed6dbe34c23e11588d130fef07f8 guix: Add codesigning functionality (Carl Dong) bac2690e6f683fcedb883fe1d32f3c33c628a141 guix: Package codesigning tools (Carl Dong) 0a2176d47767972e4cd5ed302c1dbeedece1708b guix: Reindent existing manifest.scm (Carl Dong) c090a3e9238ba2df07875b4708e908d8dca4ed9b Makefile.am: use APP_DIST_DIR instead of hard-coding dist (Carl Dong) Pull request description: This is the last PR before we reach feature-parity with the Gitian process! Note: I tried using the `Makefile` inside the distsrc to make the dmg instead of manually listing out the commands, but `make` seems to want to re-make a lot of other files which broke the dmg. The workflow looks something like this: 1. `env [ FOO=bar... ] ./contrib/guix/guix-build` (add additional env vars as necessary) 2. Codesigners only: 1. Copy `guix-build-<short-id>/output/x86_64-apple-darwin18/bitcoin-<short-id>-osx-unsigned.tar.gz` and `guix-build-<short-id>/output/x86_64-w64-mingw32/bitcoin-<short-id>-win-unsigned.tar.gz` to signing computer 2. Codesign with `./detached-sig-create.sh` inside the tarball 3. Upload contents of `signature-{osx,win}.tar.gz` to https://github.com/bitcoin-core/bitcoin-detached-sigs (as a new tag) 3. Checkout new tag for `bitcoin-core/bitcoin-detached-sigs` with the detached signatures 4. `env [ FOO=bar... ] DETACHED_SIGS_REPO=<path/to/bitcoin-detached-sigs> ./contrib/guix/guix-codesign` (modify env vars as necessary) 5. Make sure `guix.sigs` is cloned and updated 6. `env GUIX_SIGS_REPO=<path/to/guix.sigs> SIGNER=0x96AB007F1A7ED999=dongcarl ./contrib/guix/guix-attest` (modify env vars as necessary) 7. Commit your new signatures and SHA256SUMS in `guix.sigs` 8. Optionally, after there are multiple signatures in `guix.sigs`: `env GUIX_SIGS_REPO=<path/to/guix.sigs> ./contrib/guix/guix-verify` ACKs for top commit: laanwj: Tested ACK ee883201cf134952284632e9e9ae72bf1c8c792f achow101: ACK ee883201cf134952284632e9e9ae72bf1c8c792f Tree-SHA512: e812a07a5f19f900600c70cb9c717769ef544a6c0c12760b5558b76b6b37df863257f3dbf38b0757e6e06e334470267e94c9f2bdbc27409d6837b1a0bfc6acbc
2021-05-24 15:31:05 +02:00
(list clang-toolchain-10 binutils imagemagick libtiff librsvg font-tuffy cmake xorriso python-signapple))
Merge #17595: guix: Enable building for `x86_64-w64-mingw32` target a35e3235891d35daa167116cc70340140e883f06 guix: Appease travis. (Carl Dong) 0b66d22da5f53640e22f05adf880782c613e6d0f guix: Use gcc-9 for mingw-w64 instead of 8 (Carl Dong) ba0b99bdd613ba7f17c6247ece3001e1b44759a3 guix: Don't set MINGW_HAS_SECURE_API CFLAG in depends (Carl Dong) 93439a71eda49fb69f1e82966a23a946733aa6fa guix: Bump to upstream commit with mingw-w64 changes (Carl Dong) 35a96792dda9e78165b1598aeac7b2ab759e7be5 guix: Check mingw symbols, improve SSP fix docs (Carl Dong) 449d8fe25bbe25daacfc67aa89ca32b0a3254c5a guix: Expand on INT trap message (Carl Dong) 3f1f03c67a8e9edf487f08d272adb18b0a3942c8 guix: Spelling fixes (Carl Dong) ff821dd2a1c600488d11e7d9a20e9179ecc9144b guix: Reinstate make-ssp-fixed-gcc (Carl Dong) 360a9e0ad50a36ec79a1a160dbed3966689fd41c guix: Bump time-machine for mingw-w64 patches (Carl Dong) 93e41b7e3b54c17fd1b4c61ee95fc0dc2827e954 guix: Use gcc-8 for mingw-w64 instead of 7 (Carl Dong) ef4f7e4c45c60a69406134122f091c77c6ef740f guix: Set the well-known timezone env var (Carl Dong) acf4b3b3b5accf60a19441a0298ef27001b78e72 guix: Make x86_64-w64-mingw32 builds reproducible (Carl Dong) c4cce00eac691625b78b92f7dba0b7f57def19e5 guix: Remove dead links from README. (Carl Dong) df953a4c9a6143f45864757b706c88b6fa70545a guix: Appease shellcheck. (Carl Dong) 91897c95e191d293eb27d8af15cbeafc5b8f3895 guix: Improve guix-build.sh documentation (Carl Dong) 570d769c6c59b9f6d1a2b95b2ed60432cb33b3ba guix: Build support for Windows (Carl Dong) Pull request description: ~~Based on: https://github.com/bitcoin/bitcoin/pull/16519~~ Based on: #17933 (Time Machines are... shall we say... superior :grin:) This PR allows us to perform Guix builds for the `x86_64-w64-mingw32` target. We do this _without_ splitting up the build script like we do in Gitian by using this newfangled alien technology called `case` statements. (This is WIP and might be changed to `if` statements soon) ACKs for top commit: fanquake: ACK a35e3235891d35daa167116cc70340140e883f06 2/3 Tree-SHA512: c471951c23eb2cda919a71285d8b8f2580cb20f09d5db17b53e13dbd8813e01b3e7a83ea848e4913fd0f2bc12c6c133c5f76b54e65c0d89fed4dfd2e0be19875
2020-04-12 12:08:46 +02:00
(else '())))))