eb4c43e49f625895670866b89bb56ca641c4eeb7 doc: documents how to calculate m_assumed_blockchain_size and m_assumed_chain_state_size on the release process. (marcoagner) Pull request description: Regarding [this](https://github.com/bitcoin/bitcoin/pull/15183#issuecomment-463133734) on https://github.com/bitcoin/bitcoin/pull/15183. Added an "Additional information" section for this which seems reasonable to me but may not be the best place for this. Also, let me know if anything else should be documented here (like more details). ACKs for top commit: laanwj: ACK eb4c43e49f625895670866b89bb56ca641c4eeb7 Tree-SHA512: 7e6fc46740daa01dd9be5a8da7846e7a9f7fa866bf31fdc2cb252f90c698cfd6ef954f9588f7abcebda2355ec2b2a380635e14a164e53e77d38abefa3e2cc698
14 KiB
Release Process
-
Update translations, see translation_process.md.
-
Update manpages, see gen-manpages.sh.
-
Update release candidate version in
configure.ac
(CLIENT_VERSION_RC
)
Before every minor and major release:
- Update bips.md to account for changes since the last release.
- Update version in
configure.ac
(don't forget to setCLIENT_VERSION_IS_RELEASE
totrue
) (don't forget to setCLIENT_VERSION_RC
to0
) - Write release notes (see below)
- Update
src/chainparams.cpp
nMinimumChainWork with information from the getblockchaininfo rpc. - Update
src/chainparams.cpp
defaultAssumeValid with information from the getblockhash rpc.- The selected value must not be orphaned so it may be useful to set the value two blocks back from the tip.
- Testnet should be set some tens of thousands back from the tip due to reorgs there.
- This update should be reviewed with a reindex-chainstate with assumevalid=0 to catch any defect that causes rejection of blocks in the past history.
Before every major release:
- Update hardcoded seeds. TODO: Give example PR for Dash
- Update
src/chainparams.cpp
m_assumed_blockchain_size and m_assumed_chain_state_size with the current size plus some overhead (see this for information on how to calculate them). - Update
src/chainparams.cpp
chainTxData with statistics about the transaction count and rate. Use the output of the RPCgetchaintxstats
, see this pull request for an example. Reviewers can verify the results by runninggetchaintxstats <window_block_count> <window_last_block_hash>
with thewindow_block_count
andwindow_last_block_hash
from your output. - Update version of
contrib/gitian-descriptors/*.yml
: usually one'd want to do this on master after branching off the release - but be sure to at least do it before a new major release
First time / New builders
If you're using the automated script (found in contrib/gitian-build.py), then at this point you should run it with the "--setup" command. Otherwise ignore this.
Check out the source code in the following directory hierarchy.
cd /path/to/your/toplevel/build
git clone https://github.com/dashpay/gitian.sigs.git
git clone https://github.com/dashpay/dash-detached-sigs.git
git clone https://github.com/devrandom/gitian-builder.git
git clone https://github.com/dashpay/dash.git
Dash Core maintainers/release engineers, suggestion for writing release notes
Write release notes. git shortlog helps a lot, for example:
git shortlog --no-merges v(current version, e.g. 0.12.2)..v(new version, e.g. 0.12.3)
Generate list of authors:
git log --format='- %aN' v(current version, e.g. 0.16.0)..v(new version, e.g. 0.16.1) | sort -fiu
Tag version (or release candidate) in git
git tag -s v(new version, e.g. 0.12.3)
Setup and perform Gitian builds
If you're using the automated script (found in contrib/gitian-build.py), then at this point you should run it with the "--build" command. Otherwise ignore this.
Setup Gitian descriptors:
pushd ./dash
export SIGNER="(your Gitian key, ie UdjinM6, Pasta, etc)"
export VERSION=(new version, e.g. 0.12.3)
git fetch
git checkout v${VERSION}
popd
Ensure your gitian.sigs are up-to-date if you wish to gverify your builds against other Gitian signatures.
pushd ./gitian.sigs
git pull
popd
Ensure gitian-builder is up-to-date:
pushd ./gitian-builder
git pull
popd
Fetch and create inputs: (first time, or when dependency versions change)
pushd ./gitian-builder
mkdir -p inputs
wget -O inputs/osslsigncode-2.0.tar.gz https://github.com/mtrojnar/osslsigncode/archive/2.0.tar.gz
echo '5a60e0a4b3e0b4d655317b2f12a810211c50242138322b16e7e01c6fbb89d92f inputs/osslsigncode-2.0.tar.gz' | sha256sum -c
popd
Create the macOS SDK tarball, see the macOS build instructions for details, and copy it into the inputs directory.
Optional: Seed the Gitian sources cache and offline git repositories
NOTE: Gitian is sometimes unable to download files. If you have errors, try the step below.
By default, Gitian will fetch source files as needed. To cache them ahead of time, make sure you have checked out the tag you want to build in dash, then:
pushd ./gitian-builder
make -C ../dash/depends download SOURCES_PATH=`pwd`/cache/common
popd
Only missing files will be fetched, so this is safe to re-run for each build.
NOTE: Offline builds must use the --url flag to ensure Gitian fetches only from local URLs. For example:
pushd ./gitian-builder
./bin/gbuild --url dash=/path/to/dash,signature=/path/to/sigs {rest of arguments}
popd
The gbuild invocations below DO NOT DO THIS by default.
Build and sign Dash Core for Linux, Windows, and macOS:
pushd ./gitian-builder
./bin/gbuild --num-make 2 --memory 3000 --commit dash=v${VERSION} ../dash/contrib/gitian-descriptors/gitian-linux.yml
./bin/gsign --signer "$SIGNER" --release ${VERSION}-linux --destination ../gitian.sigs/ ../dash/contrib/gitian-descriptors/gitian-linux.yml
mv build/out/dash-*.tar.gz build/out/src/dash-*.tar.gz ../
./bin/gbuild --num-make 2 --memory 3000 --commit dash=v${VERSION} ../dash/contrib/gitian-descriptors/gitian-win.yml
./bin/gsign --signer "$SIGNER" --release ${VERSION}-win-unsigned --destination ../gitian.sigs/ ../dash/contrib/gitian-descriptors/gitian-win.yml
mv build/out/dash-*-win-unsigned.tar.gz inputs/dash-win-unsigned.tar.gz
mv build/out/dash-*.zip build/out/dash-*.exe ../
./bin/gbuild --num-make 2 --memory 3000 --commit dash=v${VERSION} ../dash/contrib/gitian-descriptors/gitian-osx.yml
./bin/gsign --signer "$SIGNER" --release ${VERSION}-osx-unsigned --destination ../gitian.sigs/ ../dash/contrib/gitian-descriptors/gitian-osx.yml
mv build/out/dash-*-osx-unsigned.tar.gz inputs/dash-osx-unsigned.tar.gz
mv build/out/dash-*.tar.gz build/out/dash-*.dmg ../
popd
Build output expected:
- source tarball (
dash-${VERSION}.tar.gz
) - linux 32-bit and 64-bit dist tarballs (
dash-${VERSION}-linux[32|64].tar.gz
) - windows 32-bit and 64-bit unsigned installers and dist zips (
dash-${VERSION}-win[32|64]-setup-unsigned.exe
,dash-${VERSION}-win[32|64].zip
) - macOS unsigned installer and dist tarball (
dash-${VERSION}-osx-unsigned.dmg
,dash-${VERSION}-osx64.tar.gz
) - Gitian signatures (in
gitian.sigs/${VERSION}-<linux|{win,osx}-unsigned>/(your Gitian key)/
)
Verify other gitian builders signatures to your own. (Optional)
Add other gitian builders keys to your gpg keyring, and/or refresh keys.
gpg --import dash/contrib/gitian-keys/*.pgp
gpg --refresh-keys
Verify the signatures
pushd ./gitian-builder
./bin/gverify -v -d ../gitian.sigs/ -r ${VERSION}-linux ../dash/contrib/gitian-descriptors/gitian-linux.yml
./bin/gverify -v -d ../gitian.sigs/ -r ${VERSION}-win-unsigned ../dash/contrib/gitian-descriptors/gitian-win.yml
./bin/gverify -v -d ../gitian.sigs/ -r ${VERSION}-osx-unsigned ../dash/contrib/gitian-descriptors/gitian-osx.yml
popd
Next steps:
Commit your signature to gitian.sigs:
pushd gitian.sigs
git add ${VERSION}-linux/"${SIGNER}"
git add ${VERSION}-win-unsigned/"${SIGNER}"
git add ${VERSION}-osx-unsigned/"${SIGNER}"
git commit -a
git push # Assuming you can push to the gitian.sigs tree
popd
Codesigner only: Create Windows/macOS detached signatures:
- Only one person handles codesigning. Everyone else should skip to the next step.
- Only once the Windows/macOS builds each have 3 matching signatures may they be signed with their respective release keys.
Codesigner only: Sign the macOS binary:
transfer dashcore-osx-unsigned.tar.gz to macOS for signing
tar xf dashcore-osx-unsigned.tar.gz
./detached-sig-create.sh -s "Key ID" -o runtime
Enter the keychain password and authorize the signature
Move signature-osx.tar.gz back to the gitian host
Codesigner only: Sign the windows binaries:
tar xf dashcore-win-unsigned.tar.gz
./detached-sig-create.sh -key /path/to/codesign.key
Enter the passphrase for the key when prompted
signature-win.tar.gz will be created
Codesigner only: Commit the detached codesign payloads:
cd ~/dashcore-detached-sigs
checkout the appropriate branch for this release series
rm -rf *
tar xf signature-osx.tar.gz
tar xf signature-win.tar.gz
git add -A
git commit -m "point to ${VERSION}"
git tag -s v${VERSION} HEAD
git push the current branch and new tag
Non-codesigners: wait for Windows/macOS detached signatures:
- Once the Windows/macOS builds each have 3 matching signatures, they will be signed with their respective release keys.
- Detached signatures will then be committed to the dash-detached-sigs repository, which can be combined with the unsigned apps to create signed binaries.
Create (and optionally verify) the signed macOS binary:
pushd ./gitian-builder
./bin/gbuild -i --commit signature=v${VERSION} ../dash/contrib/gitian-descriptors/gitian-osx-signer.yml
./bin/gsign --signer "$SIGNER" --release ${VERSION}-osx-signed --destination ../gitian.sigs/ ../dash/contrib/gitian-descriptors/gitian-osx-signer.yml
./bin/gverify -v -d ../gitian.sigs/ -r ${VERSION}-osx-signed ../dash/contrib/gitian-descriptors/gitian-osx-signer.yml
mv build/out/dash-osx-signed.dmg ../dash-${VERSION}-osx.dmg
popd
Create (and optionally verify) the signed Windows binaries:
pushd ./gitian-builder
./bin/gbuild -i --commit signature=v${VERSION} ../dash/contrib/gitian-descriptors/gitian-win-signer.yml
./bin/gsign --signer "$SIGNER" --release ${VERSION}-win-signed --destination ../gitian.sigs/ ../dash/contrib/gitian-descriptors/gitian-win-signer.yml
./bin/gverify -v -d ../gitian.sigs/ -r ${VERSION}-win-signed ../dash/contrib/gitian-descriptors/gitian-win-signer.yml
mv build/out/dash-*win64-setup.exe ../dash-${VERSION}-win64-setup.exe
popd
Commit your signature for the signed macOS/Windows binaries:
pushd gitian.sigs
git add ${VERSION}-osx-signed/"${SIGNER}"
git add ${VERSION}-win-signed/"${SIGNER}"
git commit -m "Add ${SIGNER} ${VERSION} signed binaries signatures"
git push # Assuming you can push to the gitian.sigs tree
popd
After 3 or more people have gitian-built and their results match:
- Create
SHA256SUMS.asc
for the builds, and GPG-sign it:
sha256sum * > SHA256SUMS
The list of files should be:
dash-${VERSION}-aarch64-linux-gnu.tar.gz
dash-${VERSION}-arm-linux-gnueabihf.tar.gz
dash-${VERSION}-i686-pc-linux-gnu.tar.gz
dash-${VERSION}-riscv64-linux-gnu.tar.gz
dash-${VERSION}-x86_64-linux-gnu.tar.gz
dash-${VERSION}-osx64.tar.gz
dash-${VERSION}-osx.dmg
dash-${VERSION}.tar.gz
dash-${VERSION}-win64-setup.exe
dash-${VERSION}-win64.zip
The *-debug*
files generated by the Gitian build contain debug symbols
for troubleshooting by developers. It is assumed that anyone that is interested
in debugging can run Gitian to generate the files for themselves. To avoid
end-user confusion about which file to pick, as well as save storage
space do not upload these to the dash.org server.
- GPG-sign it, delete the unsigned file:
gpg --digest-algo sha256 --clearsign SHA256SUMS # outputs SHA256SUMS.asc
rm SHA256SUMS
(the digest algorithm is forced to sha256 to avoid confusion of the Hash:
header that GPG adds with the SHA256 used for the files)
Note: check that SHA256SUMS itself doesn't end up in SHA256SUMS, which is a spurious/nonsensical entry.
-
Upload zips and installers, as well as
SHA256SUMS.asc
from last step, to the dash.org server -
Update dash.org
-
Announce the release:
-
Release on Dash forum: https://www.dash.org/forum/topic/official-announcements.54/
-
Optionally Discord, twitter, reddit /r/Dashpay, ... but this will usually sort out itself
-
Notify flare so that he can start building the PPAs
-
Archive release notes for the new version to
doc/release-notes/
(branchmaster
and branch of the release) -
Create a new GitHub release with a link to the archived release notes.
-
Celebrate
-
Additional information
How to calculate m_assumed_blockchain_size
and m_assumed_chain_state_size
Both variables are used as a guideline for how much space the user needs on their drive in total, not just strictly for the blockchain. Note that all values should be taken from a fully synced node and have an overhead of 5-10% added on top of its base value.
To calculate m_assumed_blockchain_size
:
- For
mainnet
-> Take the size of the Dash Core data directory, excluding/regtest
and/testnet3
directories. - For
testnet
-> Take the size of the/testnet3
directory.
To calculate m_assumed_chain_state_size
:
- For
mainnet
-> Take the size of the/chainstate
directory. - For
testnet
-> Take the size of the/testnet3/chainstate
directory.
Notes:
- When taking the size for
m_assumed_blockchain_size
, there's no need to exclude the/chainstate
directory since it's a guideline value and an overhead will be added anyway. - The expected overhead for growth may change over time, so it may not be the same value as last release; pay attention to that when changing the variables.