mirror of
https://github.com/dashpay/dash.git
synced 2024-12-24 11:32:46 +01:00
Merge bitcoin/bitcoin#21711: guix: Add full installation and usage documentation
fac4814106c796b8786dd90053513cc35142dfe5 doc/release-process: Add torrent creation details (Carl Dong) 5d24cc3d82dad6812f8370c3ccc7c2b5a6c12c11 guix/INSTALL: Guix installs init scripts in libdir (Carl Dong) 5da2ee49d5b44de803b671aedbdd14e5c1d71ea9 guix/INSTALL: Add coreutils/inotify-dir-recreate troubleshooting (Carl Dong) 318c60700b7bbb7ec09a29bf037e7c2787646be6 guix: Adapt release-process.md to new Guix process (Carl Dong) fcab35b2292f9221eaba521740e8b3b2511a8b78 guix-attest: Produce and sign normalized documents (Carl Dong) c2541fd0ca99481a5a792a8f2772925d64fb0491 guix: Overhaul README (Carl Dong) 46ce6ce3782dfbd8f9d26dc2ba0f284755e75f2d tree-wide: Rename gitian-keys to builder-keys (Carl Dong) fc4f8449f34e32b0b9ac9d218d6c3264b02467ba guix: Update various check_tools lists (Carl Dong) 263220a85c1df218431fafbda07c8b23ccc4ce4d guix: Check for a sane services database (Carl Dong) Pull request description: Based on: #21462 Keeping the README in one file so that it's easy to search through. Will add more jumping links later so navigation is easier. Current TODOs: - [x] Shell installer option: prompt user to re-login for `/etc/profile.d` entry to be picked up - [x] Binary tarball option: prompt user to create `/etc/profile.d` entry and re-login - [x] Fanquake docker option: complete section - [x] Arch Linux AUR option: prompt to start `guix-daemon-latest` unit after finishing "optional setup" section - [x] Building from source option: Insert dependency tree diagram that I made - [x] Building from source option: redo sectioning, kind of a mess right now - [x] Optional setup: make clear which parts are only needed if building from source - [x] Workaround 1 for GnuTLS: perhaps mention how to remove Guix build farm's key - [x] Overall (after everything): Make the links work. Note to self: wherever possible, tell user how to check that something is true rather than branching by installation option. ACKs for top commit: fanquake: ACK fac4814106c796b8786dd90053513cc35142dfe5 - going to go ahead and merge this now. It's a lot of documentation, and could probably be nit-picked / improved further, however, that can continue over the next few weeks. I'm sure more (backportable) improvements / clarifications will be made while we progress through RCs towards a new release. Tree-SHA512: dc46c0ecdfc67c7c7743ca26e4a603eb3f54adbf81be2f4c1f4c20577ebb84b5250b9c9ec89c0e9860337ab1c7cff94d7963c603287267deecfe1cd987fa070a
This commit is contained in:
fanquake
PastaPastaPasta
parent
5b027e6611
commit
f54ec9cde7
@ -27,8 +27,8 @@ for Debian-based Linux systems. If you compile dashd/dash-qt yourself, there are
|
||||
### [Gitian-descriptors](/contrib/gitian-descriptors) ###
|
||||
Notes on getting Gitian builds up and running using KVM.
|
||||
|
||||
### [Gitian-keys](/contrib/gitian-keys)
|
||||
PGP keys used for signing Dash Core [Gitian release](/doc/release-process.md) results.
|
||||
### [Builder keys](/contrib/builder-keys)
|
||||
PGP keys used for signing Dash Core [release](/doc/release-process.md) results.
|
||||
|
||||
### [MacDeploy](/contrib/macdeploy) ###
|
||||
Scripts and notes for Mac builds.
|
||||
|
||||
@ -3,8 +3,7 @@ PGP keys
|
||||
|
||||
This folder contains the public keys of developers and active contributors.
|
||||
|
||||
The keys are mainly used to sign git commits or the build results of Gitian
|
||||
builds.
|
||||
The keys are mainly used to sign git commits or the build results of builds.
|
||||
|
||||
You can import the keys into gpg as follows. Also, make sure to fetch the
|
||||
latest version from the key server to see if any key was revoked in the
|
||||
813
contrib/guix/INSTALL.md
Normal file
813
contrib/guix/INSTALL.md
Normal file
@ -0,0 +1,813 @@
|
||||
# Guix Installation and Setup
|
||||
|
||||
This only needs to be done once per machine. If you have already completed the
|
||||
installation and setup, please proceed to [perform a build](./README.md).
|
||||
|
||||
Otherwise, you may choose from one of the following options to install Guix:
|
||||
|
||||
1. Using the official **shell installer script** [⤓ skip to section][install-script]
|
||||
- Maintained by Guix developers
|
||||
- Easiest (automatically performs *most* setup)
|
||||
- Works on nearly all Linux distributions
|
||||
- Only installs latest release
|
||||
- Binary installation only, requires high level of trust
|
||||
- Note: The script needs to be run as root, so it should be inspected before it's run
|
||||
2. Using the official **binary tarball** [⤓ skip to section][install-bin-tarball]
|
||||
- Maintained by Guix developers
|
||||
- Normal difficulty (full manual setup required)
|
||||
- Works on nearly all Linux distributions
|
||||
- Installs any release
|
||||
- Binary installation only, requires high level of trust
|
||||
3. Using fanquake's **Docker image** [↗︎ external instructions][install-fanquake-docker]
|
||||
- Maintained by fanquake
|
||||
- Easy (automatically performs *some* setup)
|
||||
- Works wherever Docker images work
|
||||
- Installs any release
|
||||
- Binary installation only, requires high level of trust
|
||||
4. Using a **distribution-maintained package** [⤓ skip to section][install-distro-pkg]
|
||||
- Maintained by distribution's Guix package maintainer
|
||||
- Normal difficulty (manual setup required)
|
||||
- Works only on distributions with Guix packaged, see: https://repology.org/project/guix/versions
|
||||
- Installs a release decided on by package maintainer
|
||||
- Source or binary installation depending on the distribution
|
||||
5. Building **from source** [⤓ skip to section][install-source]
|
||||
- Maintained by you
|
||||
- Hard, but rewarding
|
||||
- Can be made to work on most Linux distributions
|
||||
- Installs any commit (more granular)
|
||||
- Source installation, requires lower level of trust
|
||||
|
||||
## Options 1 and 2: Using the official shell installer script or binary tarball
|
||||
|
||||
The installation instructions for both the official shell installer script and
|
||||
the binary tarballs can be found in the GNU Guix Manual's [Binary Installation
|
||||
section](https://guix.gnu.org/manual/en/html_node/Binary-Installation.html).
|
||||
|
||||
Note that running through the binary tarball installation steps is largely
|
||||
equivalent to manually performing what the shell installer script does.
|
||||
|
||||
Note that at the time of writing (July 5th, 2021), the shell installer script
|
||||
automatically creates an `/etc/profile.d` entry which the binary tarball
|
||||
installation instructions do not ask you to create. However, you will likely
|
||||
need this entry for better desktop integration. Please see [this
|
||||
section](#add-an-etcprofiled-entry) for instructions on how to add a
|
||||
`/etc/profile.d/guix.sh` entry.
|
||||
|
||||
Regardless of which installation option you chose, the changes to
|
||||
`/etc/profile.d` will not take effect until the next shell or desktop session,
|
||||
so you should log out and log back in.
|
||||
|
||||
## Option 3: Using fanquake's Docker image
|
||||
|
||||
Please refer to fanquake's instructions
|
||||
[here](https://github.com/fanquake/core-review/tree/master/guix).
|
||||
|
||||
Note that the `Dockerfile` is largely equivalent to running through the binary
|
||||
tarball installation steps.
|
||||
|
||||
## Option 4: Using a distribution-maintained package
|
||||
|
||||
Note that this section is based on the distro packaging situation at the time of
|
||||
writing (July 2021). Guix is expected to be more widely packaged over time. For
|
||||
an up-to-date view on Guix's package status/version across distros, please see:
|
||||
https://repology.org/project/guix/versions
|
||||
|
||||
### Debian 11 (Bullseye)/Ubuntu 21.04 (Hirsute Hippo)
|
||||
|
||||
Guix v1.2.0 is available as a distribution package starting in [Debian
|
||||
11](https://packages.debian.org/bullseye/guix) and [Ubuntu
|
||||
21.04](https://packages.ubuntu.com/hirsute/guix).
|
||||
|
||||
Note that if you intend on using Guix without using any substitutes (more
|
||||
details [here][security-model]), v1.2.0 has a known problems when building
|
||||
GnuTLS from source. Solutions and workarounds are documented
|
||||
[here](#gnutls-test-suite-fail-status-request-revoked).
|
||||
|
||||
|
||||
To install:
|
||||
```sh
|
||||
sudo apt install guix
|
||||
```
|
||||
|
||||
For up-to-date information on Debian and Ubuntu's release history:
|
||||
- [Debian release history](https://www.debian.org/releases/)
|
||||
- [Ubuntu release history](https://ubuntu.com/about/release-cycle)
|
||||
|
||||
### Arch Linux
|
||||
|
||||
Guix is available in the AUR as
|
||||
[`guix`](https://aur.archlinux.org/packages/guix/), please follow the
|
||||
installation instructions in the Arch Linux Wiki ([live
|
||||
link](https://wiki.archlinux.org/index.php/Guix#AUR_Package_Installation),
|
||||
[2021/03/30
|
||||
permalink](https://wiki.archlinux.org/index.php?title=Guix&oldid=637559#AUR_Package_Installation))
|
||||
to install Guix.
|
||||
|
||||
At the time of writing (2021/03/30), the `check` phase will fail if the path to
|
||||
guix's build directory is longer than 36 characters due to an anachronistic
|
||||
character limit on the shebang line. Since the `check` phase happens after the
|
||||
`build` phase, which may take quite a long time, it is recommended that users
|
||||
either:
|
||||
|
||||
1. Skip the `check` phase
|
||||
- For `makepkg`: `makepkg --nocheck ...`
|
||||
- For `yay`: `yay --mflags="--nocheck" ...`
|
||||
- For `paru`: `paru --nocheck ...`
|
||||
2. Or, check their build directory's length beforehand
|
||||
- For those building with `makepkg`: `pwd | wc -c`
|
||||
|
||||
## Option 5: Building from source
|
||||
|
||||
Building Guix from source is a rather involved process but a rewarding one for
|
||||
those looking to minimize trust and maximize customizability (e.g. building a
|
||||
particular commit of Guix). Previous experience with using autotools-style build
|
||||
systems to build packages from source will be helpful. *hic sunt dracones.*
|
||||
|
||||
I strongly urge you to at least skim through the entire section once before you
|
||||
start issuing commands, as it will save you a lot of unncessary pain and
|
||||
anguish.
|
||||
|
||||
### Installing common build tools
|
||||
|
||||
There are a few basic build tools that are required for most things we'll build,
|
||||
so let's install them now:
|
||||
|
||||
Text transformation/i18n:
|
||||
- `autopoint` (sometimes packaged in `gettext`)
|
||||
- `help2man`
|
||||
- `po4a`
|
||||
- `texinfo`
|
||||
|
||||
Build system tools:
|
||||
- `g++` w/ C++11 support
|
||||
- `libtool`
|
||||
- `autoconf`
|
||||
- `automake`
|
||||
- `pkg-config` (sometimes packaged as `pkgconf`)
|
||||
- `make`
|
||||
- `cmake`
|
||||
|
||||
Miscellaneous:
|
||||
- `git`
|
||||
- `gnupg`
|
||||
- `python3`
|
||||
|
||||
### Building and Installing Guix's dependencies
|
||||
|
||||
In order to build Guix itself from source, we need to first make sure that the
|
||||
necessary dependencies are installed and discoverable. The most up-to-date list
|
||||
of Guix's dependencies is kept in the ["Requirements"
|
||||
section](https://guix.gnu.org/manual/en/html_node/Requirements.html) of the Guix
|
||||
Reference Manual.
|
||||
|
||||
Depending on your distribution, most or all of these dependencies may already be
|
||||
packaged and installable without manually building and installing.
|
||||
|
||||
For reference, the graphic below outlines Guix v1.3.0's dependency graph:
|
||||
|
||||
|
||||
|
||||
#### Guile
|
||||
|
||||
##### Choosing a Guile version and sticking to it
|
||||
|
||||
One of the first things you need to decide is which Guile version you want to
|
||||
use: Guile v2.2 or Guile v3.0. Unlike the python2 to python3 transition, Guile
|
||||
v2.2 and Guile v3.0 are largely compatible, as evidenced by the fact that most
|
||||
Guile packages and even [Guix
|
||||
itself](https://guix.gnu.org/en/blog/2020/guile-3-and-guix/) support running on
|
||||
both.
|
||||
|
||||
What is important here is that you **choose one**, and you **remain consistent**
|
||||
with your choice throughout **all Guile-related packages**, no matter if they
|
||||
are installed via the distribution's package manager or installed from source.
|
||||
This is because the files for Guile packages are installed to directories which
|
||||
are separated based on the Guile version.
|
||||
|
||||
###### Example: Checking that Ubuntu's `guile-git` is compatible with your chosen Guile version
|
||||
|
||||
On Ubuntu Focal:
|
||||
|
||||
```sh
|
||||
$ apt show guile-git
|
||||
Package: guile-git
|
||||
...
|
||||
Depends: guile-2.2, guile-bytestructures, libgit2-dev
|
||||
...
|
||||
```
|
||||
|
||||
As you can see, the package `guile-git` depends on `guile-2.2`, meaning that it
|
||||
was likely built for Guile v2.2. This means that if you decided to use Guile
|
||||
v3.0 on Ubuntu Focal, you would need to build guile-git from source instead of
|
||||
using the distribution package.
|
||||
|
||||
On Ubuntu Hirsute:
|
||||
|
||||
```sh
|
||||
$ apt show guile-git
|
||||
Package: guile-git
|
||||
...
|
||||
Depends: guile-3.0 | guile-2.2, guile-bytestructures (>= 1.0.7-3~), libgit2-dev (>= 1.0)
|
||||
...
|
||||
```
|
||||
|
||||
In this case, `guile-git` depends on either `guile-3.0` or `guile-2.2`, meaning
|
||||
that it would work no matter what Guile version you decided to use.
|
||||
|
||||
###### Corner case: Multiple versions of Guile on one system
|
||||
|
||||
It is recommended to only install one version of Guile, so that build systems do
|
||||
not get confused about which Guile to use.
|
||||
|
||||
However, if you insist on having both Guile v2.2 and Guile v3.0 installed on
|
||||
your system, then you need to **consistently** specify one of
|
||||
`GUILE_EFFECTIVE_VERSION=3.0` or `GUILE_EFFECTIVE_VERSION=2.2` to all
|
||||
`./configure` invocations for Guix and its dependencies.
|
||||
|
||||
##### Installing Guile
|
||||
|
||||
Guile is most likely already packaged for your distribution, so after you have
|
||||
[chosen a Guile version](#choosing-a-guile-version-and-sticking-to-it), install
|
||||
it via your distribution's package manager.
|
||||
|
||||
If your distribution splits packages into `-dev`-suffixed and
|
||||
non-`-dev`-suffixed sub-packages (as is the case for Debian-derived
|
||||
distributions), please make sure to install both. For example, to install Guile
|
||||
v2.2 on Debian/Ubuntu:
|
||||
|
||||
```sh
|
||||
apt install guile-2.2 guile-2.2-dev
|
||||
```
|
||||
|
||||
#### Mixing distribution packages and source-built packages
|
||||
|
||||
At the time of writing, most distributions have _some_ of Guix's dependencies
|
||||
packaged, but not all. This means that you may want to install the distribution
|
||||
package for some dependencies, and manually build-from-source for others.
|
||||
|
||||
Distribution packages usually install to `/usr`, which is different from the
|
||||
default `./configure` prefix of source-built packages: `/usr/local`.
|
||||
|
||||
This means that if you mix-and-match distribution packages and source-built
|
||||
packages and do not specify exactly `--prefix=/usr` to `./configure` for
|
||||
source-built packages, you will need to augment the `GUILE_LOAD_PATH` and
|
||||
`GUILE_LOAD_COMPILED_PATH` environment variables so that Guile will look
|
||||
under the right prefix and find your source-built packages.
|
||||
|
||||
For example, if you are using Guile v2.2, and have Guile packages in the
|
||||
`/usr/local` prefix, either add the following lines to your `.profile` or
|
||||
`.bash_profile` so that the environment variable is properly set for all future
|
||||
shell logins, or paste the lines into a POSIX-style shell to temporarily modify
|
||||
the environment variables of your current shell session.
|
||||
|
||||
```sh
|
||||
# Help Guile v2.2.x find packages in /usr/local
|
||||
export GUILE_LOAD_PATH="/usr/local/share/guile/site/2.2${GUILE_LOAD_PATH:+:}$GUILE_LOAD_PATH"
|
||||
export GUILE_LOAD_COMPILED_PATH="/usr/local/lib/guile/2.2/site-ccache${GUILE_LOAD_COMPILED_PATH:+:}$GUILE_COMPILED_LOAD_PATH"
|
||||
```
|
||||
|
||||
Note that these environment variables are used to check for packages during
|
||||
`./configure`, so they should be set as soon as possible should you want to use
|
||||
a prefix other than `/usr`.
|
||||
|
||||
<!-- ##### Example: Consistently using Guile 3.0 on Ubuntu -->
|
||||
|
||||
<!-- For example, on Ubuntu, if you choose to use Guile 3.0 and install the -->
|
||||
<!-- `guile-3.0` package, you want to make sure that if you also want to install the -->
|
||||
<!-- `guile-git` package with `apt` that said `guile-git` package was built for Guile -->
|
||||
<!-- v3.0. This can be checked by invoking the following: -->
|
||||
|
||||
<!-- ``` -->
|
||||
<!-- apt update -->
|
||||
<!-- apt show guile-git -->
|
||||
<!-- ``` -->
|
||||
|
||||
#### Building and installing source-built packages
|
||||
|
||||
***IMPORTANT**: A few dependencies have non-obvious quirks/erratas which are documented in the
|
||||
sub-sections immediately below. Please read these sections before proceeding to
|
||||
build and install these packages.*
|
||||
|
||||
Although you should always refer to the README or INSTALL files for the most
|
||||
accurate information, most of these dependencies use autoconf-style build
|
||||
systems (check if there's a `configure.ac` file), and will likely do the right
|
||||
thing with the following:
|
||||
|
||||
Clone the repository and check out the latest release:
|
||||
```sh
|
||||
git clone <git-repo-of-dependency>/<dependency>.git
|
||||
cd <dependency>
|
||||
git tag -l # check for the latest release
|
||||
git checkout <latest-release>
|
||||
```
|
||||
|
||||
For autoconf-based build systems (if `./autogen.sh` or `configure.ac` exists at
|
||||
the root of the repository):
|
||||
|
||||
```sh
|
||||
./autogen.sh || autoreconf -vfi
|
||||
./configure --prefix=<prefix>
|
||||
make
|
||||
sudo make install
|
||||
```
|
||||
|
||||
For CMake-based build systems (if `CMakeLists.txt` exists at the root of the
|
||||
repository):
|
||||
|
||||
```sh
|
||||
mkdir build && cd build
|
||||
cmake .. -DCMAKE_INSTALL_PREFIX=<prefix>
|
||||
sudo cmake --build . --target install
|
||||
```
|
||||
|
||||
If you choose not to specify exactly `--prefix=/usr` to `./configure`, please
|
||||
make sure you've carefully read the [previous section] on mixing distribution
|
||||
packages and source-built packages.
|
||||
|
||||
##### Binding packages require `-dev`-suffixed packages
|
||||
|
||||
Relevant for:
|
||||
- Everyone
|
||||
|
||||
When building bindings, the `-dev`-suffixed version of the original package
|
||||
needs to be installed. For example, building `Guile-zlib` on Debian-derived
|
||||
distributions requires that `zlib1g-dev` is installed.
|
||||
|
||||
When using bindings, the `-dev`-suffixed version of the original package still
|
||||
needs to be installed. This is particularly problematic when distribution
|
||||
packages are mispackaged like `guile-sqlite3` is in Ubuntu Focal such that
|
||||
installing `guile-sqlite3` does not automatically install `libsqlite3-dev` as a
|
||||
dependency.
|
||||
|
||||
Below is a list of relevant Guile bindings and their corresponding `-dev`
|
||||
packages in Debian at the time of writing.
|
||||
|
||||
| Guile binding package | -dev Debian package |
|
||||
|-----------------------|---------------------|
|
||||
| guile-gcrypt | libgcrypt-dev |
|
||||
| guile-git | libgit2-dev |
|
||||
| guile-lzlib | liblz-dev |
|
||||
| guile-ssh | libssh-dev |
|
||||
| guile-sqlite3 | libsqlite3-dev |
|
||||
| guile-zlib | zlib1g-dev |
|
||||
|
||||
##### `guile-git` actually depends on `libgit2 >= 1.1`
|
||||
|
||||
Relevant for:
|
||||
- Those building `guile-git` from source against `libgit2 < 1.1`
|
||||
- Those installing `guile-git` from their distribution where `guile-git` is
|
||||
built against `libgit2 < 1.1`
|
||||
|
||||
As of v0.4.0, `guile-git` claims to only require `libgit2 >= 0.28.0`, however,
|
||||
it actually requires `libgit2 >= 1.1`, otherwise, it will be confused by a
|
||||
reference of `origin/keyring`: instead of interpreting the reference as "the
|
||||
'keyring' branch of the 'origin' remote", the reference is interpreted as "the
|
||||
branch literally named 'origin/keyring'"
|
||||
|
||||
This is especially notable because Ubuntu Focal packages `libgit2 v0.28.4`, and
|
||||
`guile-git` is built against it.
|
||||
|
||||
Should you be in this situation, you need to build both `libgit2 v1.1.x` and
|
||||
`guile-git` from source.
|
||||
|
||||
Source: http://logs.guix.gnu.org/guix/2020-11-12.log#232527
|
||||
|
||||
##### `{scheme,guile}-bytestructures` v1.0.8 and v1.0.9 are broken for Guile v2.2
|
||||
|
||||
Relevant for:
|
||||
- Those building `{scheme,guile}-bytestructures` from source against Guile v2.2
|
||||
|
||||
Commit
|
||||
[707eea3](https://github.com/TaylanUB/scheme-bytestructures/commit/707eea3a85e1e375e86702229ebf73d496377669)
|
||||
introduced a regression for Guile v2.2 and was first included in v1.0.8, this
|
||||
was later corrected in commit
|
||||
[ec9a721](https://github.com/TaylanUB/scheme-bytestructures/commit/ec9a721957c17bcda13148f8faa5f06934431ff7)
|
||||
and included in v1.1.0.
|
||||
|
||||
TL;DR If you decided to use Guile v2.2, do not use `{scheme,guile}-bytestructures` v1.0.8 or v1.0.9.
|
||||
|
||||
### Building and Installing Guix itself
|
||||
|
||||
Start by cloning Guix:
|
||||
|
||||
```
|
||||
git clone https://git.savannah.gnu.org/git/guix.git
|
||||
cd guix
|
||||
```
|
||||
|
||||
You will likely want to build the latest release, however, if the latest release
|
||||
when you're reading this is still 1.2.0 then you may want to use 95aca29 instead
|
||||
to avoid a problem in the GnuTLS test suite.
|
||||
|
||||
```
|
||||
git branch -a -l 'origin/version-*' # check for the latest release
|
||||
git checkout <latest-release>
|
||||
```
|
||||
|
||||
Bootstrap the build system:
|
||||
```
|
||||
./bootstrap
|
||||
```
|
||||
|
||||
Configure with the recommended `--localstatedir` flag:
|
||||
```
|
||||
./configure --localstatedir=/var
|
||||
```
|
||||
|
||||
Note: If you intend to hack on Guix in the future, you will need to supply the
|
||||
same `--localstatedir=` flag for all future Guix `./configure` invocations. See
|
||||
the last paragraph of this
|
||||
[section](https://guix.gnu.org/manual/en/html_node/Requirements.html) for more
|
||||
details.
|
||||
|
||||
Build Guix (this will take a while):
|
||||
```
|
||||
make -j$(nproc)
|
||||
```
|
||||
|
||||
Install Guix:
|
||||
|
||||
```
|
||||
sudo make install
|
||||
```
|
||||
|
||||
### Post-"build from source" Setup
|
||||
|
||||
#### Creating and starting a `guix-daemon-original` service with a fixed `argv[0]`
|
||||
|
||||
At this point, guix will be installed to `${bindir}`, which is likely
|
||||
`/usr/local/bin` if you did not override directory variables at
|
||||
`./configure`-time. More information on standard Automake directory variables
|
||||
can be found
|
||||
[here](https://www.gnu.org/software/automake/manual/html_node/Standard-Directory-Variables.html).
|
||||
|
||||
However, the Guix init scripts and service configurations for Upstart, systemd,
|
||||
SysV, and OpenRC are installed (in `${libdir}`) to launch
|
||||
`${localstatedir}/guix/profiles/per-user/root/current-guix/bin/guix-daemon`,
|
||||
which does not yet exist, and will only exist after [`root` performs their first
|
||||
`guix pull`](#guix-pull-as-root).
|
||||
|
||||
We need to create a `-original` version of these init scripts that's pointed to
|
||||
the binaries we just built and `make install`'ed in `${bindir}` (normally,
|
||||
`/usr/local/bin`).
|
||||
|
||||
Example for `systemd`, run as `root`:
|
||||
|
||||
```sh
|
||||
# Create guix-daemon-original.service by modifying guix-daemon.service
|
||||
libdir=# set according to your PREFIX (default is /usr/local/lib)
|
||||
bindir="$(dirname $(command -v guix-daemon))"
|
||||
sed -E -e "s|/\S*/guix/profiles/per-user/root/current-guix/bin/guix-daemon|${bindir}/guix-daemon|" "${libdir}"/systemd/system/guix-daemon.service > /etc/systemd/system/guix-daemon-original.service
|
||||
chmod 664 /etc/systemd/system/guix-daemon-original.service
|
||||
|
||||
# Make systemd recognize the new service
|
||||
systemctl daemon-reload
|
||||
|
||||
# Make sure that the non-working guix-daemon.service is stopped and disabled
|
||||
systemctl stop guix-daemon
|
||||
systemctl disable guix-daemon
|
||||
|
||||
# Make sure that the working guix-daemon-original.service is started and enabled
|
||||
systemctl enable guix-daemon-original
|
||||
systemctl start guix-daemon-original
|
||||
```
|
||||
|
||||
#### Creating `guix-daemon` users / groups
|
||||
|
||||
Please see the [relevant
|
||||
section](https://guix.gnu.org/manual/en/html_node/Build-Environment-Setup.html)
|
||||
in the Guix Reference Manual for more details.
|
||||
|
||||
## Optional setup
|
||||
|
||||
At this point, you are set up to [use Guix to build Bitcoin
|
||||
Core](./README.md#usage). However, if you want to polish your setup a bit and
|
||||
make it "what Guix intended", then read the next few subsections.
|
||||
|
||||
### Add an `/etc/profile.d` entry
|
||||
|
||||
This section definitely does not apply to you if you installed Guix using:
|
||||
1. The shell installer script
|
||||
2. fanquake's Docker image
|
||||
3. Debian's `guix` package
|
||||
|
||||
#### Background
|
||||
|
||||
Although Guix knows how to update itself and its packages, it does so in a
|
||||
non-invasive way (it does not modify `/usr/local/bin/guix`).
|
||||
|
||||
Instead, it does the following:
|
||||
|
||||
- After a `guix pull`, it updates
|
||||
`/var/guix/profiles/per-user/$USER/current-guix`, and creates a symlink
|
||||
targeting this directory at `$HOME/.config/guix/current`
|
||||
|
||||
- After a `guix install`, it updates
|
||||
`/var/guix/profiles/per-user/$USER/guix-profile`, and creates a symlink
|
||||
targeting this directory at `$HOME/.guix-profile`
|
||||
|
||||
Therefore, in order for these operations to affect your shell/desktop sessions
|
||||
(and for the principle of least astonishment to hold), their corresponding
|
||||
directories have to be added to well-known environment variables like `$PATH`,
|
||||
`$INFOPATH`, `$XDG_DATA_DIRS`, etc.
|
||||
|
||||
In other words, if `$HOME/.config/guix/current/bin` does not exist in your
|
||||
`$PATH`, a `guix pull` will have no effect on what `guix` you are using. Same
|
||||
goes for `$HOME/.guix-profile/bin`, `guix install`, and installed packages.
|
||||
|
||||
Helpfully, after a `guix pull` or `guix install`, a message will be printed like
|
||||
so:
|
||||
|
||||
```
|
||||
hint: Consider setting the necessary environment variables by running:
|
||||
|
||||
GUIX_PROFILE="$HOME/.guix-profile"
|
||||
. "$GUIX_PROFILE/etc/profile"
|
||||
|
||||
Alternately, see `guix package --search-paths -p "$HOME/.guix-profile"'.
|
||||
```
|
||||
|
||||
However, this is somewhat tedious to do for both `guix pull` and `guix install`
|
||||
for each user on the system that wants to properly use `guix`. I recommend that
|
||||
you instead add an entry to `/etc/profile.d` instead. This is done by default
|
||||
when installing the Debian package later than 1.2.0-4 and when using the shell
|
||||
script installer.
|
||||
|
||||
#### Instructions
|
||||
|
||||
Create `/etc/profile.d/guix.sh` with the following content:
|
||||
```sh
|
||||
# _GUIX_PROFILE: `guix pull` profile
|
||||
_GUIX_PROFILE="$HOME/.config/guix/current"
|
||||
if [ -L $_GUIX_PROFILE ]; then
|
||||
export PATH="$_GUIX_PROFILE/bin${PATH:+:}$PATH"
|
||||
# Export INFOPATH so that the updated info pages can be found
|
||||
# and read by both /usr/bin/info and/or $GUIX_PROFILE/bin/info
|
||||
# When INFOPATH is unset, add a trailing colon so that Emacs
|
||||
# searches 'Info-default-directory-list'.
|
||||
export INFOPATH="$_GUIX_PROFILE/share/info:$INFOPATH"
|
||||
fi
|
||||
|
||||
# GUIX_PROFILE: User's default profile
|
||||
GUIX_PROFILE="$HOME/.guix-profile"
|
||||
[ -L $GUIX_PROFILE ] || return
|
||||
GUIX_LOCPATH="$GUIX_PROFILE/lib/locale"
|
||||
export GUIX_PROFILE GUIX_LOCPATH
|
||||
|
||||
[ -f "$GUIX_PROFILE/etc/profile" ] && . "$GUIX_PROFILE/etc/profile"
|
||||
|
||||
# set XDG_DATA_DIRS to include Guix installations
|
||||
export XDG_DATA_DIRS="$GUIX_PROFILE/share:${XDG_DATA_DIRS:-/usr/local/share/:/usr/share/}"
|
||||
```
|
||||
|
||||
Please note that this will not take effect until the next shell or desktop
|
||||
session (log out and log back in).
|
||||
|
||||
### `guix pull` as root
|
||||
|
||||
Before you do this, you need to read the section on [choosing your security
|
||||
model][security-model] and adjust `guix` and `guix-daemon` flags according to
|
||||
your choice, as invoking `guix pull` may pull substitutes from substitute
|
||||
servers (which you may not want).
|
||||
|
||||
As mentioned in a previous section, Guix expects
|
||||
`${localstatedir}/guix/profiles/per-user/root/current-guix` to be populated with
|
||||
`root`'s Guix profile, `guix pull`-ed and built by some former version of Guix.
|
||||
However, this is not the case when we build from source. Therefore, we need to
|
||||
perform a `guix pull` as `root`:
|
||||
|
||||
```sh
|
||||
sudo --login guix pull --branch=version-<latest-release-version>
|
||||
# or
|
||||
sudo --login guix pull --commit=<particular-commit>
|
||||
```
|
||||
|
||||
`guix pull` is quite a long process (especially if you're using
|
||||
`--no-substitute`). If you encounter build problems, please refer to the
|
||||
[troubleshooting section](#troubleshooting).
|
||||
|
||||
Note that running a bare `guix pull` with no commit or branch specified will
|
||||
pull the latest commit on Guix's master branch, which is likely fine, but not
|
||||
recommended.
|
||||
|
||||
If you installed Guix from source, you may get an error like the following:
|
||||
```sh
|
||||
error: while creating symlink '/root/.config/guix/current' No such file or directory
|
||||
```
|
||||
To resolve this, simply:
|
||||
```
|
||||
sudo mkdir -p /root/.config/guix
|
||||
```
|
||||
Then try the `guix pull` command again.
|
||||
|
||||
After the `guix pull` finishes successfully,
|
||||
`${localstatedir}/guix/profiles/per-user/root/current-guix` should be populated.
|
||||
|
||||
#### Using the newly-pulled `guix` by restarting the daemon
|
||||
|
||||
Depending on how you installed Guix, you should now make sure that your init
|
||||
scripts and service configurations point to the newly-pulled `guix-daemon`.
|
||||
|
||||
##### If you built Guix from source
|
||||
|
||||
If you followed the instructions for [fixing argv\[0\]][fix-argv0], you can now
|
||||
do the following:
|
||||
|
||||
```sh
|
||||
systemctl stop guix-daemon-original
|
||||
systemctl disable guix-daemon-original
|
||||
|
||||
systemctl enable guix-daemon
|
||||
systemctl start guix-daemon
|
||||
```
|
||||
|
||||
##### If you installed Guix via the Debian/Ubuntu distribution packages
|
||||
|
||||
You will need to create a `guix-daemon-latest` service which points to the new
|
||||
`guix` rather than a pinned one.
|
||||
|
||||
```sh
|
||||
# Create guix-daemon-latest.service by modifying guix-daemon.service
|
||||
sed -E -e "s|/usr/bin/guix-daemon|/var/guix/profiles/per-user/root/current-guix/bin/guix-daemon|" /etc/systemd/system/guix-daemon.service > /lib/systemd/system/guix-daemon-latest.service
|
||||
chmod 664 /lib/systemd/system/guix-daemon-latest.service
|
||||
|
||||
# Make systemd recognize the new service
|
||||
systemctl daemon-reload
|
||||
|
||||
# Make sure that the old guix-daemon.service is stopped and disabled
|
||||
systemctl stop guix-daemon
|
||||
systemctl disable guix-daemon
|
||||
|
||||
# Make sure that the new guix-daemon-latest.service is started and enabled
|
||||
systemctl enable guix-daemon-latest
|
||||
systemctl start guix-daemon-latest
|
||||
```
|
||||
|
||||
##### If you installed Guix via lantw44's Arch Linux AUR package
|
||||
|
||||
At the time of writing (July 5th, 2021) the systemd unit for "updated Guix" is
|
||||
`guix-daemon-latest.service`, therefore, you should do the following:
|
||||
|
||||
```sh
|
||||
systemctl stop guix-daemon
|
||||
systemctl disable guix-daemon
|
||||
|
||||
systemctl enable guix-daemon-latest
|
||||
systemctl start guix-daemon-latest
|
||||
```
|
||||
|
||||
##### Otherwise...
|
||||
|
||||
Simply do:
|
||||
|
||||
```sh
|
||||
systemctl restart guix-daemon
|
||||
```
|
||||
|
||||
### Checking everything
|
||||
|
||||
If you followed all the steps above to make your Guix setup "prim and proper,"
|
||||
you can check that you did everything properly by running through this
|
||||
checklist.
|
||||
|
||||
1. `/etc/profile.d/guix.sh` should exist and be sourced at each shell login
|
||||
|
||||
2. `guix describe` should not print `guix describe: error: failed to determine
|
||||
origin`, but rather something like:
|
||||
|
||||
```
|
||||
Generation 38 Feb 22 2021 16:39:31 (current)
|
||||
guix f350df4
|
||||
repository URL: https://git.savannah.gnu.org/git/guix.git
|
||||
branch: version-1.2.0
|
||||
commit: f350df405fbcd5b9e27e6b6aa500da7f101f41e7
|
||||
```
|
||||
|
||||
3. `guix-daemon` should be running from `${localstatedir}/guix/profiles/per-user/root/current-guix`
|
||||
|
||||
# Troubleshooting
|
||||
|
||||
## Derivation failed to build
|
||||
|
||||
When you see a build failure like below:
|
||||
|
||||
```
|
||||
building /gnu/store/...-foo-3.6.12.drv...
|
||||
/ 'check' phasenote: keeping build directory `/tmp/guix-build-foo-3.6.12.drv-0'
|
||||
builder for `/gnu/store/...-foo-3.6.12.drv' failed with exit code 1
|
||||
build of /gnu/store/...-foo-3.6.12.drv failed
|
||||
View build log at '/var/log/guix/drvs/../...-foo-3.6.12.drv.bz2'.
|
||||
cannot build derivation `/gnu/store/...-qux-7.69.1.drv': 1 dependencies couldn't be built
|
||||
cannot build derivation `/gnu/store/...-bar-3.16.5.drv': 1 dependencies couldn't be built
|
||||
cannot build derivation `/gnu/store/...-baz-2.0.5.drv': 1 dependencies couldn't be built
|
||||
guix time-machine: error: build of `/gnu/store/...-baz-2.0.5.drv' failed
|
||||
```
|
||||
|
||||
It means that `guix` failed to build a package named `foo`, which was a
|
||||
dependency of `qux`, `bar`, and `baz`. Importantly, note that the last "failed"
|
||||
line is not necessarily the root cause, the first "failed" line is.
|
||||
|
||||
Most of the time, the build failure is due to a spurious test failure or the
|
||||
package's build system/test suite breaking when running multi-threaded. To
|
||||
rebuild _just_ this derivation in a single-threaded fashion (please don't forget
|
||||
to add other `guix` flags like `--no-substitutes` as appropriate):
|
||||
|
||||
```sh
|
||||
$ guix build --cores=1 /gnu/store/...-foo-3.6.12.drv
|
||||
```
|
||||
|
||||
If the single-threaded rebuild did not succeed, you may need to dig deeper.
|
||||
You may view `foo`'s build logs in `less` like so (please replace paths with the
|
||||
path you see in the build failure output):
|
||||
|
||||
```sh
|
||||
$ bzcat /var/log/guix/drvs/../...-foo-3.6.12.drv.bz2 | less
|
||||
```
|
||||
|
||||
`foo`'s build directory is also preserved and available at
|
||||
`/tmp/guix-build-foo-3.6.12.drv-0`. However, if you fail to build `foo` multiple
|
||||
times, it may be `/tmp/...drv-1` or `/tmp/...drv-2`. Always consult the build
|
||||
failure output for the most accurate, up-to-date information.
|
||||
|
||||
### python(-minimal): [Errno 84] Invalid or incomplete multibyte or wide character
|
||||
|
||||
This error occurs when your `$TMPDIR` (default: /tmp) exists on a filesystem
|
||||
which rejects characters not present in the UTF-8 character code set. An example
|
||||
is ZFS with the utf8only=on option set.
|
||||
|
||||
More information: https://bugs.python.org/issue37584
|
||||
|
||||
### GnuTLS: test-suite FAIL: status-request-revoked
|
||||
|
||||
*The derivation is likely identified by: `/gnu/store/vhphki5sg9xkdhh2pbc8gi6vhpfzryf0-gnutls-3.6.12.drv`*
|
||||
|
||||
This unfortunate error is most common for non-substitute builders who installed
|
||||
Guix v1.2.0. The problem stems from the fact that one of GnuTLS's tests uses a
|
||||
hardcoded certificate which expired on 2020-10-24.
|
||||
|
||||
What's more unfortunate is that this GnuTLS derivation is somewhat special in
|
||||
Guix's dependency graph and is not affected by the package transformation flags
|
||||
like `--without-tests=`.
|
||||
|
||||
The easiest solution for those encountering this problem is to install a newer
|
||||
version of Guix. However, there are ways to work around this issue:
|
||||
|
||||
#### Workaround 1: Using substitutes for this single derivation
|
||||
|
||||
If you've authorized the official Guix build farm's key (more info
|
||||
[here](./README.md#step-1-authorize-the-signing-keys)), then you can use
|
||||
substitutes just for this single derivation by invoking the following:
|
||||
|
||||
```sh
|
||||
guix build --substitute-urls="https://ci.guix.gnu.org" /gnu/store/vhphki5sg9xkdhh2pbc8gi6vhpfzryf0-gnutls-3.6.12.drv
|
||||
```
|
||||
|
||||
See [this section](./README.md#removing-authorized-keys) for instructions on how
|
||||
to remove authorized keys if you don't want to keep the build farm's key
|
||||
authorized.
|
||||
|
||||
#### Workaround 2: Temporarily setting the system clock back
|
||||
|
||||
This workaround was described [here](https://issues.guix.gnu.org/44559#5).
|
||||
|
||||
Basically:
|
||||
1. Turn off networking
|
||||
2. Turn off NTP
|
||||
3. Set system time to 2020-10-01
|
||||
4. guix build --no-substitutes /gnu/store/vhphki5sg9xkdhh2pbc8gi6vhpfzryf0-gnutls-3.6.12.drv
|
||||
5. Set system time back to accurate current time
|
||||
6. Turn NTP back on
|
||||
7. Turn networking back on
|
||||
|
||||
### coreutils: FAIL: tests/tail-2/inotify-dir-recreate
|
||||
|
||||
The inotify-dir-create test fails on "remote" filesystems such as overlayfs
|
||||
(Docker's default filesystem) due to the filesystem being mistakenly recognized
|
||||
as non-remote.
|
||||
|
||||
A relatively easy workaround to this is to make sure that a somewhat traditional
|
||||
filesystem is mounted at `/tmp` (where `guix-daemon` performs its builds). For
|
||||
Docker users, this might mean [using a volume][docker/volumes], [binding
|
||||
mounting][docker/bind-mnt] from host, or (for those with enough RAM and swap)
|
||||
[mounting a tmpfs][docker/tmpfs] using the `--tmpfs` flag.
|
||||
|
||||
Please see the following links for more details:
|
||||
|
||||
- An upstream coreutils bug has been filed: [debbugs#47940](https://debbugs.gnu.org/cgi/bugreport.cgi?bug=47940)
|
||||
- A Guix bug detailing the underlying problem has been filed: [guix-issues#47935](https://issues.guix.gnu.org/47935)
|
||||
- A commit to skip this test in Guix has been merged into the core-updates branch:
|
||||
[savannah/guix@6ba1058](https://git.savannah.gnu.org/cgit/guix.git/commit/?id=6ba1058df0c4ce5611c2367531ae5c3cdc729ab4)
|
||||
|
||||
|
||||
[install-script]: #options-1-and-2-using-the-official-shell-installer-script-or-binary-tarball
|
||||
[install-bin-tarball]: #options-1-and-2-using-the-official-shell-installer-script-or-binary-tarball
|
||||
[install-fanquake-docker]: #option-3-using-fanquakes-docker-image
|
||||
[install-distro-pkg]: #option-4-using-a-distribution-maintained-package
|
||||
[install-source]: #option-5-building-from-source
|
||||
|
||||
[fix-argv0]: #creating-and-starting-a-guix-daemon-original-service-with-a-fixed-argv0
|
||||
[security-model]: ./README.md#choosing-your-security-model
|
||||
|
||||
[docker/volumes]: https://docs.docker.com/storage/volumes/
|
||||
[docker/bind-mnt]: https://docs.docker.com/storage/bind-mounts/
|
||||
[docker/tmpfs]: https://docs.docker.com/storage/tmpfs/
|
||||
@ -9,83 +9,171 @@ downloads.
|
||||
|
||||
We achieve bootstrappability by using Guix as a functional package manager.
|
||||
|
||||
## Requirements
|
||||
# Requirements
|
||||
|
||||
Conservatively, a x86_64 machine with:
|
||||
|
||||
- 16GB of free disk space on the partition that /gnu/store will reside in
|
||||
- 8GB of free disk space per platform triple you're planning on building (see
|
||||
the `HOSTS` environment variable description)
|
||||
- 8GB of free disk space **per platform triple** you're planning on building
|
||||
(see the `HOSTS` [environment variable description][env-vars-list])
|
||||
|
||||
## Setup
|
||||
# Installation and Setup
|
||||
|
||||
### Installing Guix
|
||||
If you don't have Guix installed and set up, please follow the instructions in
|
||||
[INSTALL.md](./INSTALL.md)
|
||||
|
||||
If you're just testing this out, you can use the
|
||||
[Dockerfile][fanquake/guix-docker] for convenience. It automatically speeds up
|
||||
your builds by [using substitutes](#speeding-up-builds-with-substitute-servers).
|
||||
If you don't want this behaviour, refer to the [next
|
||||
section](#choosing-your-security-model).
|
||||
# Usage
|
||||
|
||||
Otherwise, follow the [Guix installation guide][guix/bin-install].
|
||||
If you haven't considered your security model yet, please read [the relevant
|
||||
section](#choosing-your-security-model) before proceeding to perform a build.
|
||||
|
||||
> Note: For those who like to keep their filesystems clean, Guix is designed to
|
||||
> be very standalone and _will not_ conflict with your system's package
|
||||
> manager/existing setup. It _only_ touches `/var/guix`, `/gnu`, and
|
||||
> `~/.config/guix`.
|
||||
## Making the Xcode SDK available for macOS cross-compilation
|
||||
|
||||
### Choosing your security model
|
||||
In order to perform a build for macOS (which is included in the default set of
|
||||
platform triples to build), you'll need to extract the macOS SDK tarball using
|
||||
tools found in the [`macdeploy` directory](../macdeploy/README.md).
|
||||
|
||||
Guix allows us to achieve better binary security by using our CPU time to build
|
||||
everything from scratch. However, it doesn't sacrifice user choice in pursuit of
|
||||
this: users can decide whether or not to bootstrap and to use substitutes
|
||||
(pre-built packages).
|
||||
|
||||
After installation, you may want to consider [adding substitute
|
||||
servers](#speeding-up-builds-with-substitute-servers) from which to download
|
||||
pre-built packages to speed up your build if that fits your security model (say,
|
||||
if you're just testing that this works). Substitute servers are set up by
|
||||
default if you're using the [Dockerfile][fanquake/guix-docker].
|
||||
|
||||
If you prefer not to use any substitutes, make sure to supply `--no-substitutes`
|
||||
like in the following snippet. The first build will take a while, but the
|
||||
resulting packages will be cached for future builds.
|
||||
You can then either point to the SDK using the `SDK_PATH` environment variable:
|
||||
|
||||
```sh
|
||||
export ADDITIONAL_GUIX_COMMON_FLAGS='--no-substitutes'
|
||||
# Extract the SDK tarball to /path/to/parent/dir/of/extracted/SDK/Xcode-<foo>-<bar>-extracted-SDK-with-libcxx-headers
|
||||
tar -C /path/to/parent/dir/of/extracted/SDK -xaf /path/to/Xcode-<foo>-<bar>-extracted-SDK-with-libcxx-headers.tar.gz
|
||||
|
||||
# Indicate where to locate the SDK tarball
|
||||
export SDK_PATH=/path/to/parent/dir/of/extracted/SDK
|
||||
```
|
||||
|
||||
Likewise, to perform a bootstrapped build (takes even longer):
|
||||
or extract it into `depends/SDKs`:
|
||||
|
||||
```sh
|
||||
export ADDITIONAL_GUIX_COMMON_FLAGS='--no-substitutes' ADDITIONAL_GUIX_ENVIRONMENT_FLAGS='--bootstrap'
|
||||
mkdir -p depends/SDKs
|
||||
tar -C depends/SDKs -xaf /path/to/SDK/tarball
|
||||
```
|
||||
|
||||
### Using a version of Guix with `guix time-machine` capabilities
|
||||
## Building
|
||||
|
||||
> Note: This entire section can be skipped if you are already using a version of
|
||||
> Guix that has [the `guix time-machine` command][guix/time-machine].
|
||||
*The author highly recommends at least reading over the [common usage patterns
|
||||
and examples](#common-guix-build-invocation-patterns-and-examples) section below
|
||||
before starting a build. For a full list of customization options, see the
|
||||
[recognized environment variables][env-vars-list] section.*
|
||||
|
||||
Once Guix is installed, if it doesn't have the `guix time-machine` command, pull
|
||||
the latest `guix`.
|
||||
To build Bitcoin Core reproducibly with all default options, invoke the
|
||||
following from the top of a clean repository:
|
||||
|
||||
```sh
|
||||
guix pull --max-jobs=4 # change number of jobs accordingly
|
||||
./contrib/guix/guix-build
|
||||
```
|
||||
|
||||
Make sure that you are using your current profile. (You are prompted to do this
|
||||
at the end of the `guix pull`)
|
||||
## Codesigning build outputs
|
||||
|
||||
The `guix-codesign` command attaches codesignatures (produced by codesigners) to
|
||||
existing non-codesigned outputs. Please see the [release process
|
||||
documentation](/doc/release-process.md) for more context.
|
||||
|
||||
It respects many of the same environment variable flags as `guix-build`, with 2
|
||||
crucial differences:
|
||||
|
||||
1. Since only Windows and macOS build outputs require codesigning, the `HOSTS`
|
||||
environment variable will have a sane default value of `x86_64-w64-mingw32
|
||||
x86_64-apple-darwin18` instead of all the platforms.
|
||||
2. The `guix-codesign` command ***requires*** a `DETACHED_SIGS_REPO` flag.
|
||||
* _**DETACHED_SIGS_REPO**_
|
||||
|
||||
Set the directory where detached codesignatures can be found for the current
|
||||
Bitcoin Core version being built.
|
||||
|
||||
_REQUIRED environment variable_
|
||||
|
||||
An invocation with all default options would look like:
|
||||
|
||||
```bash
|
||||
export PATH="${HOME}/.config/guix/current/bin${PATH:+:}$PATH"
|
||||
```
|
||||
env DETACHED_SIGS_REPO=<path/to/bitcoin-detached-sigs> ./contrib/guix-codesign
|
||||
```
|
||||
|
||||
## Cleaning intermediate work directories
|
||||
|
||||
By default, `guix-build` leaves all intermediate files or "work directories"
|
||||
(e.g. `depends/work`, `guix-build-*/distsrc-*`) intact at the end of a build so
|
||||
that they are available to the user (to aid in debugging, etc.). However, these
|
||||
directories usually take up a large amount of disk space. Therefore, a
|
||||
`guix-clean` convenience script is provided which cleans the current `git`
|
||||
worktree to save disk space:
|
||||
|
||||
```
|
||||
./contrib/guix/guix-clean
|
||||
```
|
||||
|
||||
|
||||
## Attesting to build outputs
|
||||
|
||||
Much like how Gitian build outputs are attested to in a `gitian.sigs`
|
||||
repository, Guix build outputs are attested to in the [`guix.sigs`
|
||||
repository](https://github.com/bitcoin-core/guix.sigs).
|
||||
|
||||
After you've cloned the `guix.sigs` repository, to attest to the current
|
||||
worktree's commit/tag:
|
||||
|
||||
```
|
||||
env GUIX_SIGS_REPO=<path/to/guix.sigs> SIGNER=<gpg-key-name> ./contrib/guix/guix-attest
|
||||
```
|
||||
|
||||
See `./contrib/guix/guix-attest --help` for more information on the various ways
|
||||
`guix-attest` can be invoked.
|
||||
|
||||
## Verifying build output attestations
|
||||
|
||||
After at least one other signer has uploaded their signatures to the `guix.sigs`
|
||||
repository:
|
||||
|
||||
```
|
||||
git -C <path/to/guix.sigs> pull
|
||||
env GUIX_SIGS_REPO=<path/to/guix.sigs> ./contrib/guix/guix-verify
|
||||
```
|
||||
|
||||
|
||||
## Common `guix-build` invocation patterns and examples
|
||||
|
||||
### Keeping caches and SDKs outside of the worktree
|
||||
|
||||
If you perform a lot of builds and have a bunch of worktrees, you may find it
|
||||
more efficient to keep the depends tree's download cache, build cache, and SDKs
|
||||
outside of the worktrees to avoid duplicate downloads and unnecessary builds. To
|
||||
help with this situation, the `guix-build` script honours the `SOURCES_PATH`,
|
||||
`BASE_CACHE`, and `SDK_PATH` environment variables and will pass them on to the
|
||||
depends tree so that you can do something like:
|
||||
|
||||
```sh
|
||||
env SOURCES_PATH="$HOME/depends-SOURCES_PATH" BASE_CACHE="$HOME/depends-BASE_CACHE" SDK_PATH="$HOME/macOS-SDKs" ./contrib/guix/guix-build
|
||||
```
|
||||
|
||||
Note that the paths that these environment variables point to **must be
|
||||
directories**, and **NOT symlinks to directories**.
|
||||
|
||||
See the [recognized environment variables][env-vars-list] section for more
|
||||
details.
|
||||
|
||||
### Building a subset of platform triples
|
||||
|
||||
Sometimes you only want to build a subset of the supported platform triples, in
|
||||
which case you can override the default list by setting the space-separated
|
||||
`HOSTS` environment variable:
|
||||
|
||||
```sh
|
||||
env HOSTS='x86_64-w64-mingw32 x86_64-apple-darwin18' ./contrib/guix/guix-build
|
||||
```
|
||||
|
||||
See the [recognized environment variables][env-vars-list] section for more
|
||||
details.
|
||||
|
||||
### Controlling the number of threads used by `guix` build commands
|
||||
|
||||
Depending on your system's RAM capacity, you may want to decrease the number of
|
||||
threads used to decrease RAM usage or vice versa.
|
||||
|
||||
By default, the scripts under `./contrib/guix` will invoke all `guix` build
|
||||
commands with `--cores="$JOBS"`. Note that `$JOBS` defaults to `$(nproc)` if not
|
||||
specified. However, astute manual readers will also notice that there is a
|
||||
`--max-jobs=` flag (which defaults to 1 if unspecified).
|
||||
specified. However, astute manual readers will also notice that `guix` build
|
||||
commands also accept a `--max-jobs=` flag (which defaults to 1 if unspecified).
|
||||
|
||||
Here is the difference between `--cores=` and `--max-jobs=`:
|
||||
|
||||
@ -124,30 +212,18 @@ packages when the dependency graph allows for it, you may want to try:
|
||||
export JOBS=1 ADDITIONAL_GUIX_COMMON_FLAGS='--max-jobs=8'
|
||||
```
|
||||
|
||||
## Usage
|
||||
See the [recognized environment variables][env-vars-list] section for more
|
||||
details.
|
||||
|
||||
### As a Tool for Deterministic Builds
|
||||
|
||||
From the top of a clean Bitcoin Core repository:
|
||||
|
||||
```sh
|
||||
./contrib/guix/guix-build
|
||||
```
|
||||
|
||||
After the build finishes successfully (check the status code please), compare
|
||||
hashes:
|
||||
|
||||
```sh
|
||||
find output/ -type f -print0 | sort -z | xargs -r0 sha256sum
|
||||
```
|
||||
|
||||
#### Recognized environment variables
|
||||
## Recognized environment variables
|
||||
|
||||
* _**HOSTS**_
|
||||
|
||||
Override the space-separated list of platform triples for which to perform a
|
||||
bootstrappable build. _(defaults to "x86\_64-linux-gnu arm-linux-gnueabihf
|
||||
aarch64-linux-gnu riscv64-linux-gnu powerpc64-linux-gnu powerpc64le-linux-gnu
|
||||
bootstrappable build.
|
||||
|
||||
_(defaults to "x86\_64-linux-gnu arm-linux-gnueabihf aarch64-linux-gnu
|
||||
riscv64-linux-gnu powerpc64-linux-gnu powerpc64le-linux-gnu
|
||||
x86\_64-w64-mingw32 x86\_64-apple-darwin18")_
|
||||
|
||||
* _**SOURCES_PATH**_
|
||||
@ -156,18 +232,27 @@ find output/ -type f -print0 | sort -z | xargs -r0 sha256sum
|
||||
depends tree. Setting this to the same directory across multiple builds of the
|
||||
depends tree can eliminate unnecessary redownloading of package sources.
|
||||
|
||||
The path that this environment variable points to **must be a directory**, and
|
||||
**NOT a symlink to a directory**.
|
||||
|
||||
* _**BASE_CACHE**_
|
||||
|
||||
Set the depends tree cache for built packages. This is passed through to the
|
||||
depends tree. Setting this to the same directory across multiple builds of the
|
||||
depends tree can eliminate unnecessary building of packages.
|
||||
|
||||
The path that this environment variable points to **must be a directory**, and
|
||||
**NOT a symlink to a directory**.
|
||||
|
||||
* _**SDK_PATH**_
|
||||
|
||||
Set the path where _extracted_ SDKs can be found. This is passed through to
|
||||
the depends tree. Note that this is should be set to the _parent_ directory of
|
||||
the actual SDK (e.g. SDK_PATH=$HOME/Downloads/macOS-SDKs instead of
|
||||
$HOME/Downloads/macOS-SDKs/Xcode-12.1-12A7403-extracted-SDK-with-libcxx-headers).
|
||||
the actual SDK (e.g. `SDK_PATH=$HOME/Downloads/macOS-SDKs` instead of
|
||||
`$HOME/Downloads/macOS-SDKs/Xcode-12.1-12A7403-extracted-SDK-with-libcxx-headers`).
|
||||
|
||||
The path that this environment variable points to **must be a directory**, and
|
||||
**NOT a symlink to a directory**.
|
||||
|
||||
* _**JOBS**_
|
||||
|
||||
@ -178,13 +263,17 @@ find output/ -type f -print0 | sort -z | xargs -r0 sha256sum
|
||||
- `make` as in `make --jobs="$JOBS"`
|
||||
- `xargs` as in `xargs -P"$JOBS"`
|
||||
|
||||
See [here](#controlling-the-number-of-threads-used-by-guix-build-commands) for
|
||||
more details.
|
||||
|
||||
_(defaults to the value of `nproc` outside the container)_
|
||||
|
||||
* _**SOURCE_DATE_EPOCH**_
|
||||
|
||||
Override the reference UNIX timestamp used for bit-for-bit reproducibility,
|
||||
the variable name conforms to [standard][r12e/source-date-epoch]. _(defaults
|
||||
to the output of `$(git log --format=%at -1)`)_
|
||||
the variable name conforms to [standard][r12e/source-date-epoch].
|
||||
|
||||
_(defaults to the output of `$(git log --format=%at -1)`)_
|
||||
|
||||
* _**V**_
|
||||
|
||||
@ -200,8 +289,7 @@ find output/ -type f -print0 | sort -z | xargs -r0 sha256sum
|
||||
|
||||
A whitespace-delimited list of URLs from which to download pre-built packages.
|
||||
A URL is only used if its signing key is authorized (refer to the [substitute
|
||||
servers section](#speeding-up-builds-with-substitute-servers) for more
|
||||
details).
|
||||
servers section](#option-1-building-with-substitutes) for more details).
|
||||
|
||||
* _**ADDITIONAL_GUIX_COMMON_FLAGS**_
|
||||
|
||||
@ -216,119 +304,169 @@ find output/ -type f -print0 | sort -z | xargs -r0 sha256sum
|
||||
Additional flags to be passed to the invocation of `guix environment` inside
|
||||
`guix time-machine`.
|
||||
|
||||
## Tips and Tricks
|
||||
# Choosing your security model
|
||||
|
||||
### Speeding up builds with substitute servers
|
||||
No matter how you installed Guix, you need to decide on your security model for
|
||||
building packages with Guix.
|
||||
|
||||
_This whole section is automatically done in the convenience
|
||||
[Dockerfiles][fanquake/guix-docker]_
|
||||
Guix allows us to achieve better binary security by using our CPU time to build
|
||||
everything from scratch. However, it doesn't sacrifice user choice in pursuit of
|
||||
this: users can decide whether or not to use **substitutes** (pre-built
|
||||
packages).
|
||||
|
||||
For those who are used to life in the fast _(and trustful)_ lane, you can
|
||||
specify [substitute servers][guix/substitutes] from which to download pre-built
|
||||
packages.
|
||||
## Option 1: Building with substitutes
|
||||
|
||||
> For those who only want to use substitutes from the official Guix build farm
|
||||
> and have authorized the build farm's signing key during Guix's installation,
|
||||
> you don't need to do anything.
|
||||
### Step 1: Authorize the signing keys
|
||||
|
||||
#### Step 1: Authorize the signing keys
|
||||
Depending on the installation procedure you followed, you may have already
|
||||
authorized the Guix build farm key. In particular, the official shell installer
|
||||
script asks you if you want the key installed, and the debian distribution
|
||||
package authorized the key during installation.
|
||||
|
||||
For the official Guix build farm at https://ci.guix.gnu.org, run as root:
|
||||
You can check the current list of authorized keys at `/etc/guix/acl`.
|
||||
|
||||
At the time of writing, a `/etc/guix/acl` with just the Guix build farm key
|
||||
authorized looks something like:
|
||||
|
||||
```lisp
|
||||
(acl
|
||||
(entry
|
||||
(public-key
|
||||
(ecc
|
||||
(curve Ed25519)
|
||||
(q #8D156F295D24B0D9A86FA5741A840FF2D24F60F7B6C4134814AD55625971B394#)
|
||||
)
|
||||
)
|
||||
(tag
|
||||
(guix import)
|
||||
)
|
||||
)
|
||||
)
|
||||
```
|
||||
|
||||
If you've determined that the official Guix build farm key hasn't been
|
||||
authorized, and you would like to authorize it, run the following as root:
|
||||
|
||||
```
|
||||
guix archive --authorize < ~root/.config/guix/current/share/guix/ci.guix.gnu.org.pub
|
||||
guix archive --authorize < /var/guix/profiles/per-user/root/current-guix/share/guix/ci.guix.gnu.org.pub
|
||||
```
|
||||
|
||||
If
|
||||
`/var/guix/profiles/per-user/root/current-guix/share/guix/ci.guix.gnu.org.pub`
|
||||
doesn't exist, try:
|
||||
|
||||
```sh
|
||||
guix archive --authorize < <PREFIX>/share/guix/ci.guix.gnu.org.pub
|
||||
```
|
||||
|
||||
Where `<PREFIX>` is likely:
|
||||
- `/usr` if you installed from a distribution package
|
||||
- `/usr/local` if you installed Guix from source and didn't supply any
|
||||
prefix-modifying flags to Guix's `./configure`
|
||||
|
||||
For dongcarl's substitute server at https://guix.carldong.io, run as root:
|
||||
|
||||
```sh
|
||||
wget -qO- 'https://guix.carldong.io/signing-key.pub' | guix archive --authorize
|
||||
```
|
||||
|
||||
#### Step 2: Specify the substitute servers
|
||||
#### Removing authorized keys
|
||||
|
||||
The official Guix build farm at https://ci.guix.gnu.org is automatically used
|
||||
unless the `--no-substitutes` flag is supplied.
|
||||
To remove previously authorized keys, simply edit `/etc/guix/acl` and remove the
|
||||
`(entry (public-key ...))` entry.
|
||||
|
||||
This can be overridden for all `guix` invocations by passing the
|
||||
`--substitute-urls` option to your invocation of `guix-daemon`. This can also be
|
||||
overridden on a call-by-call basis by passing the same `--substitute-urls`
|
||||
option to client tools such at `guix environment`.
|
||||
### Step 2: Specify the substitute servers
|
||||
|
||||
To use dongcarl's substitute server for Bitcoin Core builds after having
|
||||
[authorized his signing key](#authorize-the-signing-keys):
|
||||
Once its key is authorized, the official Guix build farm at
|
||||
https://ci.guix.gnu.org is automatically used unless the `--no-substitutes` flag
|
||||
is supplied. This default list of substitute servers is overridable both on a
|
||||
`guix-daemon` level and when you invoke `guix` commands. See examples below for
|
||||
the various ways of adding dongcarl's substitute server after having [authorized
|
||||
his signing key](#authorize-the-signing-keys).
|
||||
|
||||
Change the **default list** of substitute servers by starting `guix-daemon` with
|
||||
the `--substitute-urls` option (you will likely need to edit your init script):
|
||||
|
||||
```sh
|
||||
guix-daemon <cmd> --substitute-urls='https://guix.carldong.io https://ci.guix.gnu.org'
|
||||
```
|
||||
|
||||
Override the default list of substitute servers by passing the
|
||||
`--substitute-urls` option for invocations of `guix` commands:
|
||||
|
||||
```sh
|
||||
guix <cmd> --substitute-urls='https://guix.carldong.io https://ci.guix.gnu.org'
|
||||
```
|
||||
|
||||
For scripts under `./contrib/guix`, set the `SUBSTITUTE_URLS` environment
|
||||
variable:
|
||||
|
||||
```sh
|
||||
export SUBSTITUTE_URLS='https://guix.carldong.io https://ci.guix.gnu.org'
|
||||
```
|
||||
|
||||
## Troubleshooting
|
||||
## Option 2: Disabling substitutes on an ad-hoc basis
|
||||
|
||||
### Derivation failed to build
|
||||
|
||||
When you see a build failure like below:
|
||||
|
||||
```
|
||||
building /gnu/store/...-foo-3.6.12.drv...
|
||||
/ 'check' phasenote: keeping build directory `/tmp/guix-build-foo-3.6.12.drv-0'
|
||||
builder for `/gnu/store/...-foo-3.6.12.drv' failed with exit code 1
|
||||
build of /gnu/store/...-foo-3.6.12.drv failed
|
||||
View build log at '/var/log/guix/drvs/../...-foo-3.6.12.drv.bz2'.
|
||||
cannot build derivation `/gnu/store/...-qux-7.69.1.drv': 1 dependencies couldn't be built
|
||||
cannot build derivation `/gnu/store/...-bar-3.16.5.drv': 1 dependencies couldn't be built
|
||||
cannot build derivation `/gnu/store/...-baz-2.0.5.drv': 1 dependencies couldn't be built
|
||||
guix time-machine: error: build of `/gnu/store/...-baz-2.0.5.drv' failed
|
||||
```
|
||||
|
||||
It means that `guix` failed to build a package named `foo`, which was a
|
||||
dependency of `qux`, `bar`, and `baz`. Importantly, note that the last "failed"
|
||||
line is not necessarily the root cause, the first "failed" line is.
|
||||
|
||||
Most of the time, the build failure is due to a spurious test failure or the
|
||||
package's build system/test suite breaking when running multi-threaded. To
|
||||
rebuild _just_ this derivation in a single-threaded fashion:
|
||||
If you prefer not to use any substitutes, make sure to supply `--no-substitutes`
|
||||
like in the following snippet. The first build will take a while, but the
|
||||
resulting packages will be cached for future builds.
|
||||
|
||||
For direct invocations of `guix`:
|
||||
```sh
|
||||
$ guix build --cores=1 /gnu/store/...-foo-3.6.12.drv
|
||||
guix <cmd> --no-substitutes
|
||||
```
|
||||
|
||||
If the single-threaded rebuild did not succeed, you may need to dig deeper.
|
||||
You may view `foo`'s build logs in `less` like so (please replace paths with the
|
||||
path you see in the build failure output):
|
||||
|
||||
For the scripts under `./contrib/guix/`:
|
||||
```sh
|
||||
$ bzcat /var/log/guix/drvs/../...-foo-3.6.12.drv.bz2 | less
|
||||
export ADDITIONAL_GUIX_COMMON_FLAGS='--no-substitutes'
|
||||
```
|
||||
|
||||
`foo`'s build directory is also preserved and available at
|
||||
`/tmp/guix-build-foo-3.6.12.drv-0`. However, if you fail to build `foo` multiple
|
||||
times, it may be `/tmp/...drv-1` or `/tmp/...drv-2`. Always consult the build
|
||||
failure output for the most accurate, up-to-date information.
|
||||
## Option 3: Disabling substitutes by default
|
||||
|
||||
#### python(-minimal): [Errno 84] Invalid or incomplete multibyte or wide character
|
||||
`guix-daemon` accepts a `--no-substitutes` flag, which will make sure that,
|
||||
unless otherwise overridden by a command line invocation, no substitutes will be
|
||||
used.
|
||||
|
||||
This error occurs when your `$TMPDIR` (default: /tmp) exists on a filesystem
|
||||
which rejects characters not present in the UTF-8 character code set. An example
|
||||
is ZFS with the utf8only=on option set.
|
||||
If you start `guix-daemon` using an init script, you can edit said script to
|
||||
supply this flag.
|
||||
|
||||
More information: https://bugs.python.org/issue37584
|
||||
|
||||
## FAQ
|
||||
# Purging/Uninstalling Guix
|
||||
|
||||
### How can I trust the binary installation?
|
||||
In the extraordinarily rare case where you messed up your Guix installation in
|
||||
an irreversible way, you may want to completely purge Guix from your system and
|
||||
start over.
|
||||
|
||||
As mentioned at the bottom of [this manual page][guix/bin-install]:
|
||||
1. Uninstall Guix itself according to the way you installed it. (e.g. `sudo apt
|
||||
purge guix` for Ubuntu packaging, `sudo make uninstall` for
|
||||
built-from-source).
|
||||
2. Remove all build users and groups
|
||||
|
||||
> The binary installation tarballs can be (re)produced and verified simply by
|
||||
> running the following command in the Guix source tree:
|
||||
>
|
||||
> make guix-binary.x86_64-linux.tar.xz
|
||||
You may check for relevant users and groups using:
|
||||
|
||||
### Is Guix packaged in my operating system?
|
||||
```
|
||||
getent passwd | grep guix
|
||||
getent group | grep guix
|
||||
```
|
||||
|
||||
Guix is shipped starting with [Debian Bullseye][debian/guix-bullseye] and
|
||||
[Ubuntu 21.04 "Hirsute Hippo"][ubuntu/guix-hirsute]. Other operating systems
|
||||
are working on packaging Guix as well.
|
||||
Then, you may remove users and groups using:
|
||||
|
||||
```
|
||||
sudo userdel <user>
|
||||
sudo groupdel <group>
|
||||
```
|
||||
|
||||
3. Remove all possible Guix-related directories
|
||||
- `/var/guix/`
|
||||
- `/var/log/guix/`
|
||||
- `/gnu/`
|
||||
- `/etc/guix/`
|
||||
- `/home/*/.config/guix/`
|
||||
- `/home/*/.cache/guix/`
|
||||
- `/home/*/.guix-profile/`
|
||||
- `/root/.config/guix/`
|
||||
- `/root/.cache/guix/`
|
||||
- `/root/.guix-profile/`
|
||||
|
||||
[b17e]: http://bootstrappable.org/
|
||||
[r12e/source-date-epoch]: https://reproducible-builds.org/docs/source-date-epoch/
|
||||
@ -343,3 +481,5 @@ are working on packaging Guix as well.
|
||||
[debian/guix-bullseye]: https://packages.debian.org/bullseye/guix
|
||||
[ubuntu/guix-hirsute]: https://packages.ubuntu.com/hirsute/guix
|
||||
[fanquake/guix-docker]: https://github.com/fanquake/core-review/tree/master/guix
|
||||
|
||||
[env-vars-list]: #recognized-environment-variables
|
||||
|
||||
@ -18,7 +18,7 @@ source "$(dirname "${BASH_SOURCE[0]}")/libexec/prelude.bash"
|
||||
# Required non-builtin commands should be invokable
|
||||
################
|
||||
|
||||
check_tools cat env basename mkdir xargs find
|
||||
check_tools cat env basename mkdir diff sort
|
||||
if [ -z "$NO_SIGN" ]; then
|
||||
check_tools gpg
|
||||
fi
|
||||
@ -159,6 +159,20 @@ Hint: You may wish to remove the existing attestations and their signatures by
|
||||
EOF
|
||||
}
|
||||
|
||||
# Given a document with unix line endings (just <LF>) in stdin, make all lines
|
||||
# end in <CR><LF> and make sure there's no trailing <LF> at the end of the file.
|
||||
#
|
||||
# This is necessary as cleartext signatures are calculated on text after their
|
||||
# line endings are canonicalized.
|
||||
#
|
||||
# For more information:
|
||||
# 1. https://security.stackexchange.com/a/104261
|
||||
# 2. https://datatracker.ietf.org/doc/html/rfc4880#section-7.1
|
||||
#
|
||||
rfc4880_normalize_document() {
|
||||
sed 's/$/\r/' | head -c -2
|
||||
}
|
||||
|
||||
echo "Attesting to build outputs for version: '${VERSION}'"
|
||||
echo ""
|
||||
|
||||
@ -174,7 +188,8 @@ mkdir -p "$outsigdir"
|
||||
cat "${noncodesigned_fragments[@]}" \
|
||||
| sort -u \
|
||||
| sort -k2 \
|
||||
> "$temp_noncodesigned"
|
||||
| rfc4880_normalize_document \
|
||||
> "$temp_noncodesigned"
|
||||
if [ -e noncodesigned.SHA256SUMS ]; then
|
||||
# The SHA256SUMS already exists, make sure it's exactly what we
|
||||
# expect, error out if not
|
||||
@ -201,7 +216,9 @@ mkdir -p "$outsigdir"
|
||||
cat "${sha256sum_fragments[@]}" \
|
||||
| sort -u \
|
||||
| sort -k2 \
|
||||
> "$temp_codesigned"
|
||||
| sed 's/$/\r/' \
|
||||
| rfc4880_normalize_document \
|
||||
> "$temp_codesigned"
|
||||
if [ -e codesigned.SHA256SUMS ]; then
|
||||
# The SHA256SUMS already exists, make sure it's exactly what we
|
||||
# expect, error out if not
|
||||
@ -226,6 +243,7 @@ mkdir -p "$outsigdir"
|
||||
for i in *.SHA256SUMS; do
|
||||
if [ ! -e "$i".asc ]; then
|
||||
gpg --detach-sign \
|
||||
--digest-algo sha256 \
|
||||
--local-user "$gpg_key_name" \
|
||||
--armor \
|
||||
--output "$i".asc "$i"
|
||||
|
||||
@ -18,7 +18,7 @@ source "$(dirname "${BASH_SOURCE[0]}")/libexec/prelude.bash"
|
||||
# Required non-builtin commands should be invocable
|
||||
################
|
||||
|
||||
check_tools cat mkdir make git guix
|
||||
check_tools cat mkdir make getent curl git guix
|
||||
|
||||
################
|
||||
# GUIX_BUILD_OPTIONS should be empty
|
||||
@ -186,6 +186,29 @@ fi
|
||||
#
|
||||
# However, the internal API is likely to change more than the CLI invocation
|
||||
|
||||
################
|
||||
# Services database must have basic entries
|
||||
################
|
||||
|
||||
if ! getent services http https ftp; then
|
||||
cat << EOF
|
||||
ERR: Your system's C library can not find service database entries for at least
|
||||
one of the following services: http, https, ftp.
|
||||
|
||||
Hint: Most likely, /etc/services does not exist yet (common for docker images
|
||||
and minimal distros), or you don't have permissions to access it.
|
||||
|
||||
If /etc/services does not exist yet, you may want to install the
|
||||
appropriate package for your distro which provides it.
|
||||
|
||||
On Debian/Ubuntu: netbase
|
||||
On Arch Linux: iana-etc
|
||||
|
||||
For more information, see: getent(1), services(5)
|
||||
|
||||
EOF
|
||||
|
||||
fi
|
||||
|
||||
#########
|
||||
# SETUP #
|
||||
|
||||
@ -24,18 +24,17 @@ Before every major release:
|
||||
* Update [`src/chainparams.cpp`](/src/chainparams.cpp) m_assumed_blockchain_size and m_assumed_chain_state_size with the current size plus some overhead (see [this](#how-to-calculate-m_assumed_blockchain_size-and-m_assumed_chain_state_size) for information on how to calculate them).
|
||||
* Update `src/chainparams.cpp` chainTxData with statistics about the transaction count and rate. Use the output of the RPC `getchaintxstats`, see
|
||||
[this pull request](https://github.com/bitcoin/bitcoin/pull/12270) for an example. Reviewers can verify the results by running `getchaintxstats <window_block_count> <window_last_block_hash>` with the `window_block_count` and `window_last_block_hash` from your output.
|
||||
* Update version of `contrib/gitian-descriptors/*.yml`: usually one'd want to do this on master after branching off the release - but be sure to at least do it before a new major release
|
||||
|
||||
### First time / New builders
|
||||
|
||||
If you're using the automated script (found in [contrib/gitian-build.py](/contrib/gitian-build.py)), then at this point you should run it with the "--setup" command. Otherwise ignore this.
|
||||
Install Guix using one of the installation methods detailed in
|
||||
[contrib/guix/INSTALL.md](/contrib/guix/INSTALL.md).
|
||||
|
||||
Check out the source code in the following directory hierarchy.
|
||||
|
||||
cd /path/to/your/toplevel/build
|
||||
git clone https://github.com/dashpay/gitian.sigs.git
|
||||
git clone https://github.com/dashpay/guix.sigs.git
|
||||
git clone https://github.com/dashpay/dash-detached-sigs.git
|
||||
git clone https://github.com/devrandom/gitian-builder.git
|
||||
git clone https://github.com/dashpay/dash.git
|
||||
|
||||
### Dash Core maintainers/release engineers, suggestion for writing release notes
|
||||
@ -52,114 +51,56 @@ Tag version (or release candidate) in git
|
||||
|
||||
git tag -s v(new version, e.g. 0.12.3)
|
||||
|
||||
### Setup and perform Gitian builds
|
||||
### Setup and perform Guix builds
|
||||
|
||||
If you're using the automated script (found in [contrib/gitian-build.py](/contrib/gitian-build.py)), then at this point you should run it with the "--build" command. Otherwise ignore this.
|
||||
Checkout the Dash Core version you'd like to build:
|
||||
|
||||
Setup Gitian descriptors:
|
||||
```sh
|
||||
pushd ./dash
|
||||
export SIGNER='(your builder key, ie UdjinM6, Pasta, etc)'
|
||||
export VERSION='(new version, e.g. 20.0.0)'
|
||||
git fetch "v${VERSION}"
|
||||
git checkout "v${VERSION}"
|
||||
popd
|
||||
```
|
||||
|
||||
pushd ./dash
|
||||
export SIGNER="(your Gitian key, ie UdjinM6, Pasta, etc)"
|
||||
export VERSION=(new version, e.g. 0.12.3)
|
||||
git fetch
|
||||
git checkout v${VERSION}
|
||||
popd
|
||||
Ensure your guix.sigs are up-to-date if you wish to `guix-verify` your builds
|
||||
against other `guix-attest` signatures.
|
||||
|
||||
Ensure your gitian.sigs are up-to-date if you wish to gverify your builds against other Gitian signatures.
|
||||
```sh
|
||||
git -C ./guix.sigs pull
|
||||
```
|
||||
|
||||
pushd ./gitian.sigs
|
||||
git pull
|
||||
popd
|
||||
### Create the macOS SDK tarball: (first time, or when SDK version changes)
|
||||
|
||||
Ensure gitian-builder is up-to-date:
|
||||
Create the macOS SDK tarball, see the [macOS build
|
||||
instructions](build-osx.md#deterministic-macos-dmg-notes) for
|
||||
details.
|
||||
|
||||
pushd ./gitian-builder
|
||||
git pull
|
||||
popd
|
||||
### Build and attest to build outputs:
|
||||
|
||||
Follow the relevant Guix README.md sections:
|
||||
- [Performing a build](/contrib/guix/README.md#performing-a-build)
|
||||
- [Attesting to build outputs](/contrib/guix/README.md#attesting-to-build-outputs)
|
||||
|
||||
### Fetch and create inputs: (first time, or when dependency versions change)
|
||||
### Verify other builders' signatures to your own. (Optional)
|
||||
|
||||
pushd ./gitian-builder
|
||||
mkdir -p inputs
|
||||
wget -O inputs/osslsigncode-2.0.tar.gz https://github.com/mtrojnar/osslsigncode/archive/2.0.tar.gz
|
||||
echo '5a60e0a4b3e0b4d655317b2f12a810211c50242138322b16e7e01c6fbb89d92f inputs/osslsigncode-2.0.tar.gz' | sha256sum -c
|
||||
popd
|
||||
Add other builders keys to your gpg keyring, and/or refresh keys: See `../dash/contrib/builder-keys/README.md`.
|
||||
|
||||
Create the macOS SDK tarball, see the [macOS build instructions](build-osx.md#deterministic-macos-dmg-notes) for details, and copy it into the inputs directory.
|
||||
|
||||
### Optional: Seed the Gitian sources cache and offline git repositories
|
||||
|
||||
NOTE: Gitian is sometimes unable to download files. If you have errors, try the step below.
|
||||
|
||||
By default, Gitian will fetch source files as needed. To cache them ahead of time, make sure you have checked out the tag you want to build in dash, then:
|
||||
|
||||
pushd ./gitian-builder
|
||||
make -C ../dash/depends download SOURCES_PATH=`pwd`/cache/common
|
||||
popd
|
||||
|
||||
Only missing files will be fetched, so this is safe to re-run for each build.
|
||||
|
||||
NOTE: Offline builds must use the --url flag to ensure Gitian fetches only from local URLs. For example:
|
||||
|
||||
pushd ./gitian-builder
|
||||
./bin/gbuild --url dash=/path/to/dash,signature=/path/to/sigs {rest of arguments}
|
||||
popd
|
||||
|
||||
The gbuild invocations below <b>DO NOT DO THIS</b> by default.
|
||||
|
||||
### Build and sign Dash Core for Linux, Windows, and macOS:
|
||||
|
||||
pushd ./gitian-builder
|
||||
./bin/gbuild --num-make 2 --memory 3000 --commit dash=v${VERSION} ../dash/contrib/gitian-descriptors/gitian-linux.yml
|
||||
./bin/gsign --signer "$SIGNER" --release ${VERSION}-linux --destination ../gitian.sigs/ ../dash/contrib/gitian-descriptors/gitian-linux.yml
|
||||
mv build/out/dash-*.tar.gz build/out/src/dash-*.tar.gz ../
|
||||
|
||||
./bin/gbuild --num-make 2 --memory 3000 --commit dash=v${VERSION} ../dash/contrib/gitian-descriptors/gitian-win.yml
|
||||
./bin/gsign --signer "$SIGNER" --release ${VERSION}-win-unsigned --destination ../gitian.sigs/ ../dash/contrib/gitian-descriptors/gitian-win.yml
|
||||
mv build/out/dash-*-win-unsigned.tar.gz inputs/dash-win-unsigned.tar.gz
|
||||
mv build/out/dash-*.zip build/out/dash-*.exe ../
|
||||
|
||||
./bin/gbuild --num-make 2 --memory 3000 --commit dash=v${VERSION} ../dash/contrib/gitian-descriptors/gitian-osx.yml
|
||||
./bin/gsign --signer "$SIGNER" --release ${VERSION}-osx-unsigned --destination ../gitian.sigs/ ../dash/contrib/gitian-descriptors/gitian-osx.yml
|
||||
mv build/out/dash-*-osx-unsigned.tar.gz inputs/dash-osx-unsigned.tar.gz
|
||||
mv build/out/dash-*.tar.gz build/out/dash-*.dmg ../
|
||||
popd
|
||||
|
||||
Build output expected:
|
||||
|
||||
1. source tarball (`dash-${VERSION}.tar.gz`)
|
||||
2. linux 32-bit and 64-bit dist tarballs (`dash-${VERSION}-linux[32|64].tar.gz`)
|
||||
3. windows 32-bit and 64-bit unsigned installers and dist zips (`dash-${VERSION}-win[32|64]-setup-unsigned.exe`, `dash-${VERSION}-win[32|64].zip`)
|
||||
4. macOS unsigned installer and dist tarball (`dash-${VERSION}-osx-unsigned.dmg`, `dash-${VERSION}-osx64.tar.gz`)
|
||||
5. Gitian signatures (in `gitian.sigs/${VERSION}-<linux|{win,osx}-unsigned>/(your Gitian key)/`)
|
||||
|
||||
### Verify other gitian builders signatures to your own. (Optional)
|
||||
|
||||
Add other gitian builders keys to your gpg keyring, and/or refresh keys.
|
||||
|
||||
gpg --import dash/contrib/gitian-keys/*.pgp
|
||||
gpg --refresh-keys
|
||||
|
||||
Verify the signatures
|
||||
|
||||
pushd ./gitian-builder
|
||||
./bin/gverify -v -d ../gitian.sigs/ -r ${VERSION}-linux ../dash/contrib/gitian-descriptors/gitian-linux.yml
|
||||
./bin/gverify -v -d ../gitian.sigs/ -r ${VERSION}-win-unsigned ../dash/contrib/gitian-descriptors/gitian-win.yml
|
||||
./bin/gverify -v -d ../gitian.sigs/ -r ${VERSION}-osx-unsigned ../dash/contrib/gitian-descriptors/gitian-osx.yml
|
||||
popd
|
||||
Follow the relevant Guix README.md sections:
|
||||
- [Verifying build output attestations](/contrib/guix/README.md#verifying-build-output-attestations)
|
||||
|
||||
### Next steps:
|
||||
|
||||
Commit your signature to gitian.sigs:
|
||||
Commit your signature to guix.sigs:
|
||||
|
||||
pushd gitian.sigs
|
||||
git add ${VERSION}-linux/"${SIGNER}"
|
||||
git add ${VERSION}-win-unsigned/"${SIGNER}"
|
||||
git add ${VERSION}-osx-unsigned/"${SIGNER}"
|
||||
git commit -a
|
||||
git push # Assuming you can push to the gitian.sigs tree
|
||||
popd
|
||||
```sh
|
||||
pushd guix.sigs
|
||||
git add "${VERSION}/${SIGNER}/noncodesigned.SHA256SUMS{,.asc}"
|
||||
git commit -a
|
||||
git push # Assuming you can push to the guix.sigs tree
|
||||
popd
|
||||
```
|
||||
|
||||
Codesigner only: Create Windows/macOS detached signatures:
|
||||
- Only one person handles codesigning. Everyone else should skip to the next step.
|
||||
@ -171,7 +112,7 @@ Codesigner only: Sign the macOS binary:
|
||||
tar xf dashcore-osx-unsigned.tar.gz
|
||||
./detached-sig-create.sh -s "Key ID" -o runtime
|
||||
Enter the keychain password and authorize the signature
|
||||
Move signature-osx.tar.gz back to the gitian host
|
||||
Move signature-osx.tar.gz back to the guix-build host
|
||||
|
||||
Codesigner only: Sign the windows binaries:
|
||||
|
||||
@ -182,84 +123,69 @@ Codesigner only: Sign the windows binaries:
|
||||
|
||||
Codesigner only: Commit the detached codesign payloads:
|
||||
|
||||
cd ~/dashcore-detached-sigs
|
||||
checkout the appropriate branch for this release series
|
||||
rm -rf *
|
||||
tar xf signature-osx.tar.gz
|
||||
tar xf signature-win.tar.gz
|
||||
git add -A
|
||||
git commit -m "point to ${VERSION}"
|
||||
git tag -s v${VERSION} HEAD
|
||||
git push the current branch and new tag
|
||||
```sh
|
||||
pushd ~/dashcore-detached-sigs
|
||||
# checkout the appropriate branch for this release series
|
||||
rm -rf *
|
||||
tar xf signature-osx.tar.gz
|
||||
tar xf signature-win.tar.gz
|
||||
git add -A
|
||||
git commit -m "point to ${VERSION}"
|
||||
git tag -s "v${VERSION}" HEAD
|
||||
git push the current branch and new tag
|
||||
popd
|
||||
```
|
||||
|
||||
Non-codesigners: wait for Windows/macOS detached signatures:
|
||||
|
||||
- Once the Windows/macOS builds each have 3 matching signatures, they will be signed with their respective release keys.
|
||||
- Detached signatures will then be committed to the [dash-detached-sigs](https://github.com/dashpay/dash-detached-sigs) repository, which can be combined with the unsigned apps to create signed binaries.
|
||||
|
||||
Create (and optionally verify) the signed macOS binary:
|
||||
|
||||
pushd ./gitian-builder
|
||||
./bin/gbuild -i --commit signature=v${VERSION} ../dash/contrib/gitian-descriptors/gitian-osx-signer.yml
|
||||
./bin/gsign --signer "$SIGNER" --release ${VERSION}-osx-signed --destination ../gitian.sigs/ ../dash/contrib/gitian-descriptors/gitian-osx-signer.yml
|
||||
./bin/gverify -v -d ../gitian.sigs/ -r ${VERSION}-osx-signed ../dash/contrib/gitian-descriptors/gitian-osx-signer.yml
|
||||
mv build/out/dash-osx-signed.dmg ../dash-${VERSION}-osx.dmg
|
||||
popd
|
||||
|
||||
Create (and optionally verify) the signed Windows binaries:
|
||||
|
||||
pushd ./gitian-builder
|
||||
./bin/gbuild -i --commit signature=v${VERSION} ../dash/contrib/gitian-descriptors/gitian-win-signer.yml
|
||||
./bin/gsign --signer "$SIGNER" --release ${VERSION}-win-signed --destination ../gitian.sigs/ ../dash/contrib/gitian-descriptors/gitian-win-signer.yml
|
||||
./bin/gverify -v -d ../gitian.sigs/ -r ${VERSION}-win-signed ../dash/contrib/gitian-descriptors/gitian-win-signer.yml
|
||||
mv build/out/dash-*win64-setup.exe ../dash-${VERSION}-win64-setup.exe
|
||||
popd
|
||||
Create (and optionally verify) the codesigned outputs:
|
||||
- [Codesigning](/contrib/guix/README.md#codesigning)
|
||||
|
||||
Commit your signature for the signed macOS/Windows binaries:
|
||||
|
||||
pushd gitian.sigs
|
||||
git add ${VERSION}-osx-signed/"${SIGNER}"
|
||||
git add ${VERSION}-win-signed/"${SIGNER}"
|
||||
git commit -m "Add ${SIGNER} ${VERSION} signed binaries signatures"
|
||||
git push # Assuming you can push to the gitian.sigs tree
|
||||
popd
|
||||
```sh
|
||||
pushd ./guix.sigs
|
||||
git add "${VERSION}/${SIGNER}"/all.SHA256SUMS{,.asc}
|
||||
git commit -m "Add ${SIGNER} ${VERSION} signed binaries signatures"
|
||||
git push # Assuming you can push to the guix.sigs tree
|
||||
popd
|
||||
```
|
||||
|
||||
### After 3 or more people have gitian-built and their results match:
|
||||
### After 3 or more people have guix-built and their results match:
|
||||
|
||||
- Create `SHA256SUMS.asc` for the builds, and GPG-sign it:
|
||||
Combine `all.SHA256SUMS` and `all.SHA256SUMS.asc` into a clear-signed
|
||||
`SHA256SUMS.asc` message:
|
||||
|
||||
```sh
|
||||
echo -e "-----BEGIN PGP SIGNED MESSAGE-----\nHash: SHA256\n\n$(cat all.SHA256SUMS)\n$(cat filename.txt.asc)" > SHA256SUMS.asc
|
||||
```
|
||||
|
||||
Here's an equivalent, more readable command if you're confident that you won't
|
||||
mess up whitespaces when copy-pasting:
|
||||
|
||||
```bash
|
||||
sha256sum * > SHA256SUMS
|
||||
cat << EOF > SHA256SUMS.asc
|
||||
-----BEGIN PGP SIGNED MESSAGE-----
|
||||
Hash: SHA256
|
||||
|
||||
$(cat all.SHA256SUMS)
|
||||
$(cat all.SHA256SUMS.asc)
|
||||
EOF
|
||||
```
|
||||
|
||||
The list of files should be:
|
||||
```
|
||||
dash-${VERSION}-aarch64-linux-gnu.tar.gz
|
||||
dash-${VERSION}-riscv64-linux-gnu.tar.gz
|
||||
dash-${VERSION}-x86_64-linux-gnu.tar.gz
|
||||
dash-${VERSION}-osx64.tar.gz
|
||||
dash-${VERSION}-osx.dmg
|
||||
dash-${VERSION}.tar.gz
|
||||
dash-${VERSION}-win64-setup.exe
|
||||
dash-${VERSION}-win64.zip
|
||||
```
|
||||
The `*-debug*` files generated by the Gitian build contain debug symbols
|
||||
for troubleshooting by developers. It is assumed that anyone that is interested
|
||||
in debugging can run Gitian to generate the files for themselves. To avoid
|
||||
end-user confusion about which file to pick, as well as save storage
|
||||
space *do not upload these to the dash.org server*.
|
||||
- Upload to the dash.org server:
|
||||
1. The contents of /dash/guix-build-${VERSION}/output`, except for
|
||||
`*-debug*` files.
|
||||
|
||||
- GPG-sign it, delete the unsigned file:
|
||||
```
|
||||
gpg --digest-algo sha256 --clearsign SHA256SUMS # outputs SHA256SUMS.asc
|
||||
rm SHA256SUMS
|
||||
```
|
||||
(the digest algorithm is forced to sha256 to avoid confusion of the `Hash:` header that GPG adds with the SHA256 used for the files)
|
||||
Note: check that SHA256SUMS itself doesn't end up in SHA256SUMS, which is a spurious/nonsensical entry.
|
||||
|
||||
- Upload zips and installers, as well as `SHA256SUMS.asc` from last step, to the dash.org server
|
||||
|
||||
- Update dash.org
|
||||
The `*-debug*` files generated by the guix build contain debug symbols
|
||||
for troubleshooting by developers. It is assumed that anyone that is
|
||||
interested in debugging can run guix to generate the files for
|
||||
themselves. To avoid end-user confusion about which file to pick, as well
|
||||
as save storage space *do not upload these to the dash.org server*.
|
||||
2. The combined clear-signed message you just created `SHA256SUMS.asc`
|
||||
|
||||
- Announce the release:
|
||||
|
||||
|
||||
Loading…
Reference in New Issue
Block a user