neobytes-project/neobytes
neobytes-project/neobytes
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases 1 Wiki Activity

Clarifying offline build process using gbuild --url and noting it is not

Browse Source 
done automatically.

At some point along the line, fully offline builds were no longer happening
when strictly following the release-process.md instructions.

We should ensure that users who might want to torify or build offline need
to take extra steps to remain offline.

Also, corrections to build process: including gverify examples for new builders.
This commit is contained in:
Midnight Magic 2015-09-17 14:17:26 -07:00
parent 83f0e22497
commit c73a8ea271
No known key found for this signature in database
GPG Key ID: 28DF2724F241D8EE
2 changed files with 107 additions and 22 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

58
doc/gitian-building.md
View File

@ -331,9 +331,10 @@ Getting and building the inputs
--------------------------------
Follow the instructions in [doc/release-process.md](release-process.md#fetch-and-build-inputs-first-time-or-when-dependency-versions-change)
in the bitcoin repository to install sources which require manual intervention. Also follow
the next step: 'Seed the Gitian sources cache', which will fetch all the necessary source
files to allow gitian to work offline.
in the bitcoin repository under 'Fetch and build inputs' to install sources which require
manual intervention. Also optionally follow the next step: 'Seed the Gitian sources cache
and offline git repositories' which will fetch the remaining files required for building
offline.
Building Bitcoin
----------------
@ -391,6 +392,57 @@ COMMIT=2014_03_windows_unicode_path
./bin/gbuild --commit bitcoin=${COMMIT} --url bitcoin=${URL} ../bitcoin/contrib/gitian-descriptors/gitian-osx.yml
```
Building fully offline
-----------------------
For building fully offline including attaching signatures to unsigned builds, the detached-sigs repository
and the bitcoin git repository with the desired tag must both be available locally, and then gbuild must be
told where to find them. It also requires an apt-cacher-ng which is fully-populated but set to offline mode, or
manually disabling gitian-builder's use of apt-get to update the VM build environment.
To configure apt-cacher-ng as an offline cacher, you will need to first populate its cache with the relevant
files. You must additionally patch target-bin/bootstrap-fixup to set its apt sources to something other than
plain archive.ubuntu.com: us.archive.ubuntu.com works.
So, if you use LXC:
```bash
export PATH="$PATH":/path/to/gitian-builder/libexec
export USE_LXC=1
cd /path/to/gitian-builder
./libexec/make-clean-vm --suite precise --arch amd64
LXC_ARCH=amd64 LXC_SUITE=precise on-target -u root apt-get update
LXC_ARCH=amd64 LXC_SUITE=precise on-target -u root \
-e DEBIAN_FRONTEND=noninteractive apt-get --no-install-recommends -y install \
$( sed -ne '/^packages:/,/[^-] .*/ {/^- .*/{s/"//g;s/- //;p}}' ../bitcoin/contrib/gitian-descriptors/*|sort|uniq )
LXC_ARCH=amd64 LXC_SUITE=precise on-target -u root apt-get -q -y purge grub
LXC_ARCH=amd64 LXC_SUITE=precise on-target -u root -e DEBIAN_FRONTEND=noninteractive apt-get -y dist-upgrade
```
And then set offline mode for apt-cacher-ng:
```
/etc/apt-cacher-ng/acng.conf
[...]
Offlinemode: 1
[...]
service apt-cacher-ng restart
```
Then when building, override the remote URLs that gbuild would otherwise pull from the gitian descriptors::
```bash
cd /some/root/path/
git clone https://github.com/bitcoin/bitcoin-detached-sigs.git
BTCPATH=/some/root/path/bitcoin.git
SIGPATH=/some/root/path/bitcoin-detached-sigs.git
./bin/gbuild --url bitcoin=${BTCPATH},signature=${SIGPATH} ../bitcoin/contrib/gitian-descriptors/gitian-win-signer.yml
```
Signing externally
-------------------

67
doc/release-process.md
View File

@ -6,39 +6,54 @@ Release Process
* * *
###update (commit) version in sources
###first time only or for new builders, check out the source in the following directory hierarchy
cd /path/to/your/toplevel/build
git clone https://github.com/bitcoin/gitian.sigs.git
git clone https://github.com/devrandom/gitian-builder.git
git clone https://github.com/bitcoin/bitcoin.git
###for bitcoin maintainers/release engineers, update (commit) version in sources
pushd ./bitcoin
contrib/verifysfbinaries/verify.sh
doc/README*
share/setup.nsi
src/clientversion.h (change CLIENT_VERSION_IS_RELEASE to true)
###tag version in git
###for bitcoin maintainers/release engineers, tag version in git
git tag -s v(new version, e.g. 0.8.0)
###write release notes. git shortlog helps a lot, for example:
###for bitcoin maintainers/release engineers, write release notes. git shortlog helps a lot, for example:
git shortlog --no-merges v(current version, e.g. 0.7.2)..v(new version, e.g. 0.8.0)
popd
* * *
###update gitian
###update gitian, gitian.sigs, checkout bitcoin version, and perform gitian builds
In order to take advantage of the new caching features in gitian, be sure to update to a recent version (`e9741525c` or later is recommended)
###perform gitian builds
From a directory containing the bitcoin source, gitian-builder and gitian.sigs
To ensure your gitian descriptors are accurate for direct reference for gbuild, below, run the following from a directory containing the bitcoin source:
pushd ./bitcoin
export SIGNER=(your gitian key, ie bluematt, sipa, etc)
export VERSION=(new version, e.g. 0.8.0)
pushd ./bitcoin
git checkout v${VERSION}
popd
pushd ./gitian-builder
###fetch and build inputs: (first time, or when dependency versions change)
Ensure your gitian.sigs are up-to-date if you wish to gverify your builds against other gitian signatures:
pushd ./gitian.sigs
git pull
popd
Ensure your gitian-builder sources are up-to-date to take advantage of the new caching features of gitian (`e9741525c` or later is recommended)
pushd ./gitian-builder
git pull
###fetch and create inputs: (first time, or when dependency versions change)
mkdir -p inputs
wget -P inputs https://bitcoincore.org/cfields/osslsigncode-Backports-to-1.7.1.patch
@ -52,28 +67,44 @@ Release Process
tar -C /Volumes/Xcode/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/ -czf MacOSX10.9.sdk.tar.gz MacOSX10.9.sdk
###Optional: Seed the Gitian sources cache
###Optional: Seed the Gitian sources cache and offline git repositories
By default, gitian will fetch source files as needed. For offline builds, they can be fetched ahead of time:
By default, gitian will fetch source files as needed. To cache them ahead of time:
make -C ../bitcoin/depends download SOURCES_PATH=`pwd`/cache/common
Only missing files will be fetched, so this is safe to re-run for each build.
###Build Bitcoin Core for Linux, Windows, and OS X:
Clone the detached-sigs repository:
popd
git clone https://github.com/bitcoin/bitcoin-detached-sigs.git
pushd ./bitcoin-builder
NOTE: Offline builds must use the --url flag to ensure gitian fetches only from local URLs.
For example: ./bin/bguild --url bitcoin=/path/to/bitcoin,signature=/path/to/sigs {rest of arguments}
The following gbuild invocations DO NOT DO THIS by default.
###Build (and optionally verify) Bitcoin Core for Linux, Windows, and OS X:
./bin/gbuild --commit bitcoin=v${VERSION} ../bitcoin/contrib/gitian-descriptors/gitian-linux.yml
./bin/gsign --signer $SIGNER --release ${VERSION}-linux --destination ../gitian.sigs/ ../bitcoin/contrib/gitian-descriptors/gitian-linux.yml
./bin/gverify -v -d ../gitian.sigs/ -r ${VERSION}-linux ../bitcoin/contrib/gitian-descriptors/gitian-linux.yml
mv build/out/bitcoin-*.tar.gz build/out/src/bitcoin-*.tar.gz ../
./bin/gbuild --commit bitcoin=v${VERSION} ../bitcoin/contrib/gitian-descriptors/gitian-win.yml
./bin/gsign --signer $SIGNER --release ${VERSION}-win-unsigned --destination ../gitian.sigs/ ../bitcoin/contrib/gitian-descriptors/gitian-win.yml
./bin/gverify -v -d ../gitian.sigs/ -r ${VERSION}-win-unsigned ../bitcoin/contrib/gitian-descriptors/gitian-win.yml
mv build/out/bitcoin-*-win-unsigned.tar.gz inputs/bitcoin-win-unsigned.tar.gz
mv build/out/bitcoin-*.zip build/out/bitcoin-*.exe ../
./bin/gbuild --commit bitcoin=v${VERSION} ../bitcoin/contrib/gitian-descriptors/gitian-osx.yml
./bin/gsign --signer $SIGNER --release ${VERSION}-osx-unsigned --destination ../gitian.sigs/ ../bitcoin/contrib/gitian-descriptors/gitian-osx.yml
./bin/gverify -v -d ../gitian.sigs/ -r ${VERSION}-osx-unsigned ../bitcoin/contrib/gitian-descriptors/gitian-osx.yml
mv build/out/bitcoin-*-osx-unsigned.tar.gz inputs/bitcoin-osx-unsigned.tar.gz
mv build/out/bitcoin-*.tar.gz build/out/bitcoin-*.dmg ../
popd
Build output expected:
1. source tarball (bitcoin-${VERSION}.tar.gz)
@ -98,19 +129,21 @@ Commit your signature to gitian.sigs:
Once the Windows/OSX builds each have 3 matching signatures, they will be signed with their respective release keys.
Detached signatures will then be committed to the bitcoin-detached-sigs repository, which can be combined with the unsigned apps to create signed binaries.
Create the signed OSX binary:
Create (and optionally verify) the signed OSX binary:
pushd ./gitian-builder
./bin/gbuild -i --commit signature=v${VERSION} ../bitcoin/contrib/gitian-descriptors/gitian-osx-signer.yml
./bin/gsign --signer $SIGNER --release ${VERSION}-osx-signed --destination ../gitian.sigs/ ../bitcoin/contrib/gitian-descriptors/gitian-osx-signer.yml
./bin/gverify -v -d ../gitian.sigs/ -r ${VERSION}-osx-signed ../bitcoin/contrib/gitian-descriptors/gitian-osx-signer.yml
mv build/out/bitcoin-osx-signed.dmg ../bitcoin-${VERSION}-osx.dmg
popd
Create the signed Windows binaries:
Create (and optionally verify) the signed Windows binaries:
pushd ./gitian-builder
./bin/gbuild -i --commit signature=v${VERSION} ../bitcoin/contrib/gitian-descriptors/gitian-win-signer.yml
./bin/gsign --signer $SIGNER --release ${VERSION}-win-signed --destination ../gitian.sigs/ ../bitcoin/contrib/gitian-descriptors/gitian-win-signer.yml
./bin/gverify -v -d ../gitian.sigs/ -r ${VERSION}-win-signed ../bitcoin/contrib/gitian-descriptors/gitian-win-signer.yml
mv build/out/bitcoin-*win64-setup.exe ../bitcoin-${VERSION}-win64-setup.exe
mv build/out/bitcoin-*win32-setup.exe ../bitcoin-${VERSION}-win32-setup.exe
popd