diff --git a/doc/gitian-building.md b/doc/gitian-building.md index 630b3c04a7..169727adc0 100644 --- a/doc/gitian-building.md +++ b/doc/gitian-building.md @@ -5,7 +5,7 @@ Gitian building Gitian is the deterministic build process that is used to build the Bitcoin Core executables. It provides a way to be reasonably sure that the -executables are really built from source on GitHub. It also makes sure that +executables are really built from the source on GitHub. It also makes sure that the same, tested dependencies are used and statically built into the executable. Multiple developers build the source code by following a specific descriptor @@ -13,8 +13,8 @@ Multiple developers build the source code by following a specific descriptor These results are compared and only if they match, the build is accepted and uploaded to bitcoin.org. -More independent gitian builders are needed, which is why I wrote this -guide. It is preferred to follow these steps yourself instead of using someone else's +More independent gitian builders are needed, which is why this guide exists. +It is preferred you follow these steps yourself instead of using someone else's VM image to avoid 'contaminating' the build. Table of Contents @@ -39,46 +39,46 @@ This guide explains how to set up the environment, and how to start the builds. Debian Linux was chosen as the host distribution because it has a lightweight install (in contrast to Ubuntu) and is readily available. Any kind of virtualization can be used, for example: -- [VirtualBox](https://www.virtualbox.org/), covered by this guide +- [VirtualBox](https://www.virtualbox.org/) (covered by this guide) - [KVM](http://www.linux-kvm.org/page/Main_Page) - [LXC](https://linuxcontainers.org/), see also [Gitian host docker container](https://github.com/gdm85/tenku/tree/master/docker/gitian-bitcoin-host/README.md). -You can also install on actual hardware instead of using virtualization. +You can also install gitian on actual hardware instead of using virtualization. Create a new VirtualBox VM --------------------------- In the VirtualBox GUI click "Create" and choose the following parameters in the wizard: -![](gitian-building/create_vm_page1.png) +![](gitian-building/create_new_vm.png) -- Type: Linux, Debian (64 bit) +- Type: Linux, Debian (64-bit) ![](gitian-building/create_vm_memsize.png) -- Memory Size: at least 1024MB, anything lower will really slow the build down +- Memory Size: at least 1024MB, anything less will really slow down the build. -![](gitian-building/create_vm_hard_drive.png) +![](gitian-building/create_vm_hard_disk.png) -- Hard Drive: Create a virtual hard drive now +- Hard Disk: Create a virtual hard disk now -![](gitian-building/create_vm_hard_drive_file_type.png) +![](gitian-building/create_vm_hard_disk_file_type.png) -- Hard Drive file type: Use the default, VDI (VirtualBox Disk Image) +- Hard Disk file type: Use the default, VDI (VirtualBox Disk Image) -![](gitian-building/create_vm_storage_physical_hard_drive.png) +![](gitian-building/create_vm_storage_physical_hard_disk.png) -- Storage on Physical hard drive: Dynamically Allocated +- Storage on physical hard disk: Dynamically Allocated ![](gitian-building/create_vm_file_location_size.png) -- Disk size: at least 40GB; as low as 20GB *may* be possible, but better to err on the safe side -- Push the `Create` button +- File location and size: at least 40GB; as low as 20GB *may* be possible, but better to err on the safe side +- Click `Create` -Get the [Debian 7.8 net installer](http://cdimage.debian.org/cdimage/archive/7.8.0/amd64/iso-cd/debian-7.8.0-amd64-netinst.iso) (a more recent minor version should also work, see also [Debian Network installation](https://www.debian.org/CD/netinst/)). +Get the [Debian 8.1 net installer](http://cdimage.debian.org/debian-cd/8.1.0/amd64/iso-cd/debian-8.1.0-amd64-netinst.iso) (a more recent minor version should also work, see also [Debian Network installation](https://www.debian.org/CD/netinst/)). This DVD image can be validated using a SHA256 hashing tool, for example on Unixy OSes by entering the following in a terminal: - echo "e39c36d6adc0fd86c6edb0e03e22919086c883b37ca194d063b8e3e8f6ff6a3a debian-7.8.0-amd64-netinst.iso" | sha256sum -c + echo "5d0a1f804d73aee73eee7efbb38456390558094fd19894a573f1514ca44347e0 debian-8.1.0-amd64-netinst.iso" | sha256sum -c # (must return OK) After creating the VM, we need to configure it. @@ -115,8 +115,9 @@ This section will explain how to install Debian on the newly created VM. ![](gitian-building/debian_install_1_boot_menu.png) -**Note**: Navigation in the Debian installer: To keep a setting at the default -and proceed, just press `Enter`. To select a different button, press `Tab`. +**Note**: Navigating in the Debian installer: +To keep a setting at the default and proceed, just press `Enter`. +To select a different button, press `Tab`. - Choose locale and keyboard settings (doesn't matter, you can just go with the defaults or select your own information) @@ -126,7 +127,7 @@ and proceed, just press `Enter`. To select a different button, press `Tab`. - The VM will detect network settings using DHCP, this should all proceed automatically - Configure the network: - - System name `debian`. + - Hostname `debian`. - Leave domain name empty. ![](gitian-building/debian_install_5_configure_the_network.png) @@ -136,6 +137,7 @@ and proceed, just press `Enter`. To select a different button, press `Tab`. ![](gitian-building/debian_install_6a_set_up_root_password.png) - Name the new user `debian` (the full name doesn't matter, you can leave it empty) +- Set the account username as `debian` ![](gitian-building/debian_install_7_set_up_user_fullname.png) ![](gitian-building/debian_install_8_set_up_username.png) @@ -158,13 +160,9 @@ and proceed, just press `Enter`. To select a different button, press `Tab`. ![](gitian-building/debian_install_12_choose_disk.png) - - Partitioning scheme: All files in one partition - -![](gitian-building/debian_install_13_partition_scheme.png) - - Finish partitioning and write changes to disk -> *Yes* (`Tab`, `Enter` to select the `Yes` button) -![](gitian-building/debian_install_14_finish.png) +![](gitian-building/debian_install_14_finish.png) ![](gitian-building/debian_install_15_write_changes.png) - The base system will be installed, this will take a minute or so @@ -172,51 +170,79 @@ and proceed, just press `Enter`. To select a different button, press `Tab`. ![](gitian-building/debian_install_16_choose_a_mirror.png) -- Enter proxy information (unless you are on an intranet, you can leave this empty) +- Enter proxy information (unless you are on an intranet, leave this empty) ![](gitian-building/debian_install_18_proxy_settings.png) - Wait a bit while 'Select and install software' runs - Participate in popularity contest -> *No* -- Choose software to install. We need just the base system. - -![](gitian-building/debian_install_19_software_selection.png) - +- Choose software to install. We need just the base system. - Make sure only 'SSH server' and 'Standard System Utilities' are checked - Uncheck 'Debian Desktop Environment' and 'Print Server' +![](gitian-building/debian_install_19_software_selection.png) + +- Install the GRUB boot loader to the master boot record? -> Yes + ![](gitian-building/debian_install_20_install_grub.png) -- Install the GRUB boot loader to the master boot record? -> Yes +- Device for boot loader installation -> ata-VBOX_HARDDISK -![](gitian-building/debian_install_21_finish_installation.png) +![](gitian-building/debian_install_21_install_grub_bootloader.png) - Installation Complete -> *Continue* - After installation, the VM will reboot and you will have a working Debian VM. Congratulations! +![](gitian-building/debian_install_22_finish_installation.png) + + +After Installation +------------------- +The next step in the guide involves logging in as root via SSH. +SSH login for root users is disabled by default, so we'll enable that now. + +Login to the VM using username `root` and the root password you choose earlier. +You'll be presented with a screen similar to this. + +![](gitian-building/debian_root_login.png) + +Type: + +``` +sed -i 's/^PermitRootLogin.*/PermitRootLogin yes/' /etc/ssh/sshd_config +``` +and press enter. Then, +``` +/etc/init.d/ssh restart +``` +and enter to restart SSH. Logout by typing 'logout' and pressing 'enter'. + Connecting to the VM ---------------------- After the VM has booted you can connect to it using SSH, and files can be copied from and to the VM using a SFTP utility. Connect to `localhost`, port `22222` (or the port configured when installing the VM). -On Windows you can use putty[1] and WinSCP[2]. +On Windows you can use [putty](http://www.chiark.greenend.org.uk/~sgtatham/putty/download.html) and [WinSCP](http://winscp.net/eng/index.php). -For example to connect as `root` from a Linux command prompt use +For example, to connect as `root` from a Linux command prompt use $ ssh root@localhost -p 22222 The authenticity of host '[localhost]:22222 ([127.0.0.1]:22222)' can't be established. - ECDSA key fingerprint is 8e:71:f9:5b:62:46:de:44:01:da:fb:5f:34:b5:f2:18. + RSA key fingerprint is ae:f5:c8:9f:17:c6:c7:1b:c2:1b:12:31:1d:bb:d0:c7. Are you sure you want to continue connecting (yes/no)? yes - Warning: Permanently added '[localhost]:22222' (ECDSA) to the list of known hosts. + Warning: Permanently added '[localhost]:22222' (RSA) to the list of known hosts. root@localhost's password: (enter root password configured during install) - Linux debian 3.2.0-4-amd64 #1 SMP Debian 3.2.54-2 x86_64 + + The programs included with the Debian GNU/Linux system are free software; + the exact distribution terms for each program are described in the + individual files in /usr/share/doc/*/copyright. + + Debian GNU/Linux comes with ABSOLUTELY NO WARRANTY, to the extent + permitted by applicable law. root@debian:~# Replace `root` with `debian` to log in as user. -[1] http://www.chiark.greenend.org.uk/~sgtatham/putty/download.html -[2] http://winscp.net/eng/index.php - Setting up Debian for gitian building -------------------------------------- @@ -226,13 +252,10 @@ First we need to log in as `root` to set up dependencies and make sure that our user can use the sudo command. Type/paste the following in the terminal: ```bash -apt-get install git ruby sudo apt-cacher-ng qemu-utils debootstrap lxc python-cheetah parted kpartx bridge-utils +apt-get install git ruby sudo apt-cacher-ng qemu-utils debootstrap lxc python-cheetah parted kpartx bridge-utils make ubuntu-archive-keyring adduser debian sudo ``` -When you get a colorful screen with a question about the 'LXC directory', just -go with the default (`/var/lib/lxc`). - Then set up LXC and the rest with the following, which is a complex jumble of settings and workarounds: ```bash @@ -255,7 +278,7 @@ reboot ``` At the end the VM is rebooted to make sure that the changes take effect. The steps in this -section need only to be performed once. +section only need to be performed once. Installing gitian ------------------ @@ -300,26 +323,26 @@ cd gitian-builder bin/make-base-vm --lxc --arch amd64 --suite precise ``` -There will be a lot of warnings printed during build of the image. These can be ignored. +There will be a lot of warnings printed during the build of the image. These can be ignored. **Note**: When sudo asks for a password, enter the password for the user *debian* not for *root*. Getting and building the inputs -------------------------------- -Follow the instructions in [doc/release-process.md](release-process.md) in the bitcoin repository -under 'Fetch and build inputs' to install sources which require manual intervention. Also follow -the next step: 'Seed the Gitian sources cache', which will fetch all necessary source files allowing -for gitian to work offline. +Follow the instructions in [doc/release-process.md](release-process.md#fetch-and-build-inputs-first-time-or-when-dependency-versions-change) +in the bitcoin repository to install sources which require manual intervention. Also follow +the next step: 'Seed the Gitian sources cache', which will fetch all the necessary source +files to allow gitian to work offline. Building Bitcoin ---------------- To build Bitcoin (for Linux, OSX and Windows) just follow the steps under 'perform -gitian builds' in [doc/release-process.md](release-process.md) in the bitcoin repository. +gitian builds' in [doc/release-process.md](release-process.md#perform-gitian-builds) in the bitcoin repository. -This may take a long time as it also builds the dependencies needed for each descriptor. -These dependencies will be cached after a successful build to avoid rebuilding them where possible. +This may take some time as it will build all the dependencies needed for each descriptor. +These dependencies will be cached after a successful build to avoid rebuilding them when possible. At any time you can check the package installation and build progress with @@ -331,13 +354,13 @@ tail -f var/build.log Output from `gbuild` will look something like Initialized empty Git repository in /home/debian/gitian-builder/inputs/bitcoin/.git/ - remote: Reusing existing pack: 35606, done. - remote: Total 35606 (delta 0), reused 0 (delta 0) - Receiving objects: 100% (35606/35606), 26.52 MiB | 4.28 MiB/s, done. - Resolving deltas: 100% (25724/25724), done. + remote: Counting objects: 57959, done. + remote: Total 57959 (delta 0), reused 0 (delta 0), pack-reused 57958 + Receiving objects: 100% (57959/57959), 53.76 MiB | 484.00 KiB/s, done. + Resolving deltas: 100% (41590/41590), done. From https://github.com/bitcoin/bitcoin ... (new tags, new branch etc) - --- Building for precise x86_64 --- + --- Building for precise amd64 --- Stopping target if it is up Making a new image copy stdin: is not a tty diff --git a/doc/gitian-building/create_new_vm.png b/doc/gitian-building/create_new_vm.png new file mode 100644 index 0000000000..dd22428e17 Binary files /dev/null and b/doc/gitian-building/create_new_vm.png differ diff --git a/doc/gitian-building/create_vm_hard_disk.png b/doc/gitian-building/create_vm_hard_disk.png new file mode 100644 index 0000000000..8e29816fab Binary files /dev/null and b/doc/gitian-building/create_vm_hard_disk.png differ diff --git a/doc/gitian-building/create_vm_hard_disk_file_type.png b/doc/gitian-building/create_vm_hard_disk_file_type.png new file mode 100644 index 0000000000..a157211cf5 Binary files /dev/null and b/doc/gitian-building/create_vm_hard_disk_file_type.png differ diff --git a/doc/gitian-building/create_vm_hard_drive.png b/doc/gitian-building/create_vm_hard_drive.png deleted file mode 100644 index a1706e14fd..0000000000 Binary files a/doc/gitian-building/create_vm_hard_drive.png and /dev/null differ diff --git a/doc/gitian-building/create_vm_hard_drive_file_type.png b/doc/gitian-building/create_vm_hard_drive_file_type.png deleted file mode 100644 index 251b8ee3e2..0000000000 Binary files a/doc/gitian-building/create_vm_hard_drive_file_type.png and /dev/null differ diff --git a/doc/gitian-building/create_vm_memsize.png b/doc/gitian-building/create_vm_memsize.png index 33717867a5..5abfee5337 100644 Binary files a/doc/gitian-building/create_vm_memsize.png and b/doc/gitian-building/create_vm_memsize.png differ diff --git a/doc/gitian-building/create_vm_page1.png b/doc/gitian-building/create_vm_page1.png deleted file mode 100644 index edaebc6223..0000000000 Binary files a/doc/gitian-building/create_vm_page1.png and /dev/null differ diff --git a/doc/gitian-building/create_vm_storage_physical_hard_disk.png b/doc/gitian-building/create_vm_storage_physical_hard_disk.png new file mode 100644 index 0000000000..cee16a6c63 Binary files /dev/null and b/doc/gitian-building/create_vm_storage_physical_hard_disk.png differ diff --git a/doc/gitian-building/create_vm_storage_physical_hard_drive.png b/doc/gitian-building/create_vm_storage_physical_hard_drive.png deleted file mode 100644 index 987efaa40c..0000000000 Binary files a/doc/gitian-building/create_vm_storage_physical_hard_drive.png and /dev/null differ diff --git a/doc/gitian-building/debian_install_10_configure_clock.png b/doc/gitian-building/debian_install_10_configure_clock.png index 467c79018e..7cda038ae4 100644 Binary files a/doc/gitian-building/debian_install_10_configure_clock.png and b/doc/gitian-building/debian_install_10_configure_clock.png differ diff --git a/doc/gitian-building/debian_install_11_partition_disks.png b/doc/gitian-building/debian_install_11_partition_disks.png index 18110734df..2a648c517f 100644 Binary files a/doc/gitian-building/debian_install_11_partition_disks.png and b/doc/gitian-building/debian_install_11_partition_disks.png differ diff --git a/doc/gitian-building/debian_install_12_choose_disk.png b/doc/gitian-building/debian_install_12_choose_disk.png index a00d4abf17..0f3acc498e 100644 Binary files a/doc/gitian-building/debian_install_12_choose_disk.png and b/doc/gitian-building/debian_install_12_choose_disk.png differ diff --git a/doc/gitian-building/debian_install_13_partition_scheme.png b/doc/gitian-building/debian_install_13_partition_scheme.png deleted file mode 100644 index 2f80f19b63..0000000000 Binary files a/doc/gitian-building/debian_install_13_partition_scheme.png and /dev/null differ diff --git a/doc/gitian-building/debian_install_14_finish.png b/doc/gitian-building/debian_install_14_finish.png index 411d457e95..c8ef0b37ad 100644 Binary files a/doc/gitian-building/debian_install_14_finish.png and b/doc/gitian-building/debian_install_14_finish.png differ diff --git a/doc/gitian-building/debian_install_15_write_changes.png b/doc/gitian-building/debian_install_15_write_changes.png index f26093982c..d8de00dec6 100644 Binary files a/doc/gitian-building/debian_install_15_write_changes.png and b/doc/gitian-building/debian_install_15_write_changes.png differ diff --git a/doc/gitian-building/debian_install_16_choose_a_mirror.png b/doc/gitian-building/debian_install_16_choose_a_mirror.png index d2c2e9523b..0bd985b38c 100644 Binary files a/doc/gitian-building/debian_install_16_choose_a_mirror.png and b/doc/gitian-building/debian_install_16_choose_a_mirror.png differ diff --git a/doc/gitian-building/debian_install_17_choose_a_mirror2.png b/doc/gitian-building/debian_install_17_choose_a_mirror2.png deleted file mode 100644 index cef2db0781..0000000000 Binary files a/doc/gitian-building/debian_install_17_choose_a_mirror2.png and /dev/null differ diff --git a/doc/gitian-building/debian_install_18_proxy_settings.png b/doc/gitian-building/debian_install_18_proxy_settings.png index 24ba25c109..2c19919f64 100644 Binary files a/doc/gitian-building/debian_install_18_proxy_settings.png and b/doc/gitian-building/debian_install_18_proxy_settings.png differ diff --git a/doc/gitian-building/debian_install_19_software_selection.png b/doc/gitian-building/debian_install_19_software_selection.png index d462757aff..5430456b14 100644 Binary files a/doc/gitian-building/debian_install_19_software_selection.png and b/doc/gitian-building/debian_install_19_software_selection.png differ diff --git a/doc/gitian-building/debian_install_1_boot_menu.png b/doc/gitian-building/debian_install_1_boot_menu.png index 27fd849b4f..216502e1c6 100644 Binary files a/doc/gitian-building/debian_install_1_boot_menu.png and b/doc/gitian-building/debian_install_1_boot_menu.png differ diff --git a/doc/gitian-building/debian_install_20_install_grub.png b/doc/gitian-building/debian_install_20_install_grub.png index de4f9be0c9..d853c15871 100644 Binary files a/doc/gitian-building/debian_install_20_install_grub.png and b/doc/gitian-building/debian_install_20_install_grub.png differ diff --git a/doc/gitian-building/debian_install_21_install_grub_bootloader.png b/doc/gitian-building/debian_install_21_install_grub_bootloader.png new file mode 100644 index 0000000000..493ab806a6 Binary files /dev/null and b/doc/gitian-building/debian_install_21_install_grub_bootloader.png differ diff --git a/doc/gitian-building/debian_install_21_finish_installation.png b/doc/gitian-building/debian_install_22_finish_installation.png similarity index 99% rename from doc/gitian-building/debian_install_21_finish_installation.png rename to doc/gitian-building/debian_install_22_finish_installation.png index b967c3550d..7c4445585b 100644 Binary files a/doc/gitian-building/debian_install_21_finish_installation.png and b/doc/gitian-building/debian_install_22_finish_installation.png differ diff --git a/doc/gitian-building/debian_install_2_select_a_language.png b/doc/gitian-building/debian_install_2_select_a_language.png index 1c9e0bcfc1..0228ae2c01 100644 Binary files a/doc/gitian-building/debian_install_2_select_a_language.png and b/doc/gitian-building/debian_install_2_select_a_language.png differ diff --git a/doc/gitian-building/debian_install_3_select_location.png b/doc/gitian-building/debian_install_3_select_location.png index 005c395656..7b18fba975 100644 Binary files a/doc/gitian-building/debian_install_3_select_location.png and b/doc/gitian-building/debian_install_3_select_location.png differ diff --git a/doc/gitian-building/debian_install_4_configure_keyboard.png b/doc/gitian-building/debian_install_4_configure_keyboard.png index 580c8af7c5..8e46117de4 100644 Binary files a/doc/gitian-building/debian_install_4_configure_keyboard.png and b/doc/gitian-building/debian_install_4_configure_keyboard.png differ diff --git a/doc/gitian-building/debian_install_5_configure_the_network.png b/doc/gitian-building/debian_install_5_configure_the_network.png index a7fdffc66b..8e3720f243 100644 Binary files a/doc/gitian-building/debian_install_5_configure_the_network.png and b/doc/gitian-building/debian_install_5_configure_the_network.png differ diff --git a/doc/gitian-building/debian_install_6a_set_up_root_password.png b/doc/gitian-building/debian_install_6a_set_up_root_password.png index 31bd210f38..dcade11967 100644 Binary files a/doc/gitian-building/debian_install_6a_set_up_root_password.png and b/doc/gitian-building/debian_install_6a_set_up_root_password.png differ diff --git a/doc/gitian-building/debian_install_7_set_up_user_fullname.png b/doc/gitian-building/debian_install_7_set_up_user_fullname.png index bffc6ccd7a..6763c6e08a 100644 Binary files a/doc/gitian-building/debian_install_7_set_up_user_fullname.png and b/doc/gitian-building/debian_install_7_set_up_user_fullname.png differ diff --git a/doc/gitian-building/debian_install_8_set_up_username.png b/doc/gitian-building/debian_install_8_set_up_username.png index 9e2750ad4e..bb04de96d2 100644 Binary files a/doc/gitian-building/debian_install_8_set_up_username.png and b/doc/gitian-building/debian_install_8_set_up_username.png differ diff --git a/doc/gitian-building/debian_install_9_user_password.png b/doc/gitian-building/debian_install_9_user_password.png index a26d30cba5..981f1181d7 100644 Binary files a/doc/gitian-building/debian_install_9_user_password.png and b/doc/gitian-building/debian_install_9_user_password.png differ diff --git a/doc/gitian-building/debian_root_login.png b/doc/gitian-building/debian_root_login.png new file mode 100644 index 0000000000..14cdd5ba5b Binary files /dev/null and b/doc/gitian-building/debian_root_login.png differ diff --git a/doc/gitian-building/network_settings.png b/doc/gitian-building/network_settings.png index 1d9b6428a7..9e714fd154 100644 Binary files a/doc/gitian-building/network_settings.png and b/doc/gitian-building/network_settings.png differ diff --git a/doc/gitian-building/port_forwarding_rules.png b/doc/gitian-building/port_forwarding_rules.png index e45c9efffc..9e1fa2af20 100644 Binary files a/doc/gitian-building/port_forwarding_rules.png and b/doc/gitian-building/port_forwarding_rules.png differ diff --git a/doc/gitian-building/select_startup_disk.png b/doc/gitian-building/select_startup_disk.png index 729b368fd1..5acdc3fe10 100644 Binary files a/doc/gitian-building/select_startup_disk.png and b/doc/gitian-building/select_startup_disk.png differ