e4d366788bc2e8dce8e6ca572fce08d913d15d6b build: Drop needless EXTRA_DIST content (Hennadii Stepanov)
6c4da59f5b5b3c40526d38965d4ffa7fd59f2ebc build: Drop SOURCEDIST reordering (Hennadii Stepanov)
5e6b8b391243016cb06e9e107c2e6a13a744b31e build: Use git archive as source tarball (Hennadii Stepanov)
Pull request description:
This PR:
- is an alternative to #17104
- closes#16734
- closes#6753
The idea is clear described by some developers:
- [MarcoFalke](https://github.com/bitcoin/bitcoin/pull/17097#issuecomment-540691850):
> This whole concept of explicitly listing each and every file manually (or with a fragile wildcard) is an obvious sisyphean task. I'd say all we need to do is run git archive and be done with it forever, see #16734, #6753, #11530 ...
- [laanwj](https://github.com/bitcoin/bitcoin/pull/17097#issuecomment-540706025):
> I agree, I've never been a fan of it. I don't think we have any files in the git repository we don't want to ship in the source tarball.
---
The suggested changes have a downside which is pointed by [**luke-jr**](https://github.com/bitcoin/bitcoin/pull/17104#issuecomment-540828045):
> ... but the distfile needs to include autogen-generated files.
This means that a user is not able to run `./configure && make` right away. One must run `./autogen.sh` at first.
Here are opinions about mandatory use of `./autogen.sh`:
- [ryanofsky](https://github.com/bitcoin/bitcoin/issues/16734#issuecomment-534139356):
> It's probably ok to require autogen. I think historically configure scripts were supposed to work on obscure unix systems that would just have a generic shell + make tool + c compiler, and not necessarily need gnu packages like m4 which are needed for autogen.
- [laanwj](https://github.com/bitcoin/bitcoin/issues/16734#issuecomment-540729483):
> I also think it's fine to require autogen. What is one dependency more, if you're building from source.
---
~Also this PR provides Windows users with ZIP archives of the sources. Additionally the commit ID is stored in these ZIP files as a file comment:~
---
Note for reviewers: please verify is `git archive` output deterministic?
ACKs for top commit:
MarcoFalke:
re-ACK e4d366788bc2e8dce8e6ca572fce08d913d15d6b, only change is adding two dots in a the path 🛳
laanwj:
ACK e4d366788bc2e8dce8e6ca572fce08d913d15d6b
Tree-SHA512: d1153d3ca4a580696019b92be3555ab004d197d9a2146aacff9d3150eb7093b7d40eebd6eea12d861d93ff62d62b68706e04e64dbe5ea796ff6757486e462193
fae9084ac5b10f94bdee54853d307838c4254e9c build: Skip i686 build by default in guix and gitian (MarcoFalke)
fa55a2554c2661b8f2a759044d5ac85c9979d9ca depends: Remove reference to win32 (MarcoFalke)
Pull request description:
Closes#17504
Now that we no longer provide downloads for i686 on our website (https://bitcoincore.org/en/download/), there is no need to build them by default.
i686 can still be built in depends (tested by ci/travis) and in guix/gitian by setting the appropriate `HOSTS`.
ACKs for top commit:
practicalswift:
ACK fae9084ac5b10f94bdee54853d307838c4254e9c -- patch looks correct
dongcarl:
ACK fae9084ac5b10f94bdee54853d307838c4254e9c patch looks correct
laanwj:
Code review ACK fae9084ac5b10f94bdee54853d307838c4254e9c
hebasto:
ACK fae9084ac5b10f94bdee54853d307838c4254e9c, I have reviewed the code and it looks OK, I agree it can be merged.
Tree-SHA512: b000c19a2cd2a596a52028fa298c4022c24cfdfc1bdb3795a90916d0a00a32e4dd22278db93790b6a11724e08ea8451f4f05c77bc40d1664518e11a8c82d6e29
fae75306bac4c82dd07a1b85ce5dfb020e052fe8 scripted-diff: Set gitian arch back to amd64 (MarcoFalke)
Pull request description:
This was required to allow gitian builds on non-amd64 architecture, however, it seems to break the current builds (with lxc), see https://github.com/bitcoin/bitcoin/pull/17409#issuecomment-554099626
Also, the gititan builds wouldn't be deterministic across arches anyway, see #17468
So instead of wasting more time on this, revert the change and hope that guix allows to compile on non-amd64 architectures.
Top commit has no ACKs.
Tree-SHA512: 801e9a30ae1b0882ef45d5eb3a3cf80f3ace3b99db046069dbd95b6162119e977e3cf3134287d1ac5d09483906206acc71e1ac34d6b74dbc533d46aaf73f5cc2
333362991c5219e6d9ede5fa54328bcbb0277cf2 doc: Explain $LIB in LD_PRELOAD in gitian descriptors (MarcoFalke)
fab9850ef4a76e4f09278909a2865e5175661653 scripted-diff: Avoid hardcoded libfaketime dir in gitian (MarcoFalke)
Pull request description:
Without this gitian prints warnings for me:
```
ERROR: ld.so: object '/usr/lib/x86_64-linux-gnu/faketime/libfaketime.so.1' from LD_PRELOAD cannot be preloaded (cannot open shared object file): ignored.
```
```
$ ls /usr/lib/aarch64-linux-gnu/faketime/libfaketime.so.1
/usr/lib/aarch64-linux-gnu/faketime/libfaketime.so.1
```
ACKs for top commit:
laanwj:
ACK 333362991c5219e6d9ede5fa54328bcbb0277cf2
Tree-SHA512: 3e7c4471b69c2ae38c29d0cc0db8b9eae0912085299d7f5ac67eeb4b6a2fdc7eb23d806eeeae0b0c2da22d6d1ba82513cab23652876b97aada9928b2c7d38e7e
9d1f971c7554bd5eb80792a1b9a692e0145ed516 gitian: Put things in the right place to begin with (Carl Dong)
71949a97a7d050d4cd511731062a9ffe471ac438 gitian: Eliminate rename dependency (Carl Dong)
999a9a5f5b4165dbdcb032c8d58c0ac1085ddcd1 gitian: Smaller diff with gitian-linux.yml (Carl Dong)
c4a3c25ba11b20871699e570fc7b98a7b8472503 gitian: Fix README inclusion in archives (Carl Dong)
93cb974980c578c459c15fd28e565c9722a1627d gitian: Use split-debug.sh for Win builds (Carl Dong)
Pull request description:
It would seem that our `gitian-win.yml` has not been keeping up with `gitian-linux.yml`, this PR:
1. Minimizes the diff size between `gitian-{win,linux}.yml`
2. Eliminates the `rename` dependency
ACKs for top commit:
laanwj:
ACK 9d1f971c7554bd5eb80792a1b9a692e0145ed516
Tree-SHA512: 84ed47c685e12d0064c02811907ae3d0fd3c47db8773d497dcc38f0defbfb3040fd82899fb026cf355f229b906d05a1c8038a95642bb90d044afbc2e0b239af2
993aa414d3 scripted-diff: gitian: Use REFERENCE_DATETIME directly. (Carl Dong)
Pull request description:
Fixes regression introduced by #16141.
```
-BEGIN VERIFY SCRIPT-
sed -i 's#\$REFERENCE_DATE\\\\\\ \$REFERENCE_TIME#\$REFERENCE_DATETIME#g' contrib/gitian-descriptors/*
-END VERIFY SCRIPT-
```
-----
Note that this could have been fixed by escaping properly, but using `REFERENCE_DATETIME` directly is simpler.
Future note: `REFERENCE_{DATE{,DATETIME},TIME}` is a bit ridiculous. At the very _least_ gitian should use epoch, as it is the most parse-able, and preferably set SOURCE_DATE_EPOCH.
ACKs for commit 993aa4:
Tree-SHA512: 8457e5fffde66e1d2b846547b6807416b884c171f63569f76dfefd498d2a58ad6f9eb93931eb6cfc7ff38c6b460b0c488ca87d1a68bc630c48f365a74b6ee163
b8705a091565d4373e0e8ad4aa764cfd906708db Set 'distro' explicitly (Hennadii Stepanov)
Pull request description:
The [gitian-builder](https://github.com/devrandom/gitian-builder) implicitly uses `ubuntu` as a default distro.
[bin/gbuild#L237](81edd2fc8e/bin/gbuild (L237)):
```ruby
distro = build_desc["distro"] || "ubuntu"
```
This PR sets a gitian building distro explicitly in description files.
Tree-SHA512: d2a692047f3466a5c637433610854d1100fe68a78fb03b4a81c70911fa14b0228d9cf25bcf115930aab9cc8c5063dacaf79bcd151f1c3f6fee6419389aefcb8b
eafd2593674b6890418a50eadf69425071b824e4 build: Add NX workaround for RV64 (Wladimir J. van der Laan)
f6e42256fe8900ed90f9644906b77f97e1600952 build: Allow export of environ symbols (Wladimir J. van der Laan)
Pull request description:
This export was introduced in #17270 which added
```
//! Necessary on some platforms
extern char** environ;
```
This should (finally) make the gitian build pass again (fix issue #17525.).
Built on top of #17538 which should be merged first.
Top commit has no ACKs.
Tree-SHA512: 5c2054d52d0957aec3dc945b76d8e219187d22dc03889e7a88fb76049bf8e4a3e9f4da00dd1e9dd0351211f8e70d1a1b8ad7244f0348dab698e9d14b9d0c0bd4
32da92bdf6bb55d6d312b0f85797d439cc942db5 gitian: Improve error handling (Wladimir J. van der Laan)
Pull request description:
Improve error handling in gitian builds:
- Set fail-on-error and pipefail flag, this causes a command to fail when either of the pipe stages fails, not only when the last of the stages fails, so this improves error detection.
- Also use `xargs` instead of `find -exec`, because `find` will not propagate errors in the executed command, but `xargs` will.
This will avoid some issues like #15541 where non-determinism is silently introduced due to errors caused by environment conditions (such as lack of disk space in that case).
Tree-SHA512: d5d3f22ce2d04a75e5c25e935744327c3adc704c2d303133f2918113573a564dff3d3243d5569a2b93ee7eb0e97f8e1b1ba81767e966af9015ea711a14091035
c516c3a770e93dcd283dc4a9275cba88221d0e3a [contrib] Support ARM and RISC-V symbol check (Chun Kuan Lee)
Pull request description:
Solve the TODO in the gitian-descripter
Tree-SHA512: 8115e2958af3dde43d9d9d05f0b1b1b93b1c2aa513e771a3e4e1342a5d78af2b0e40c0bbb7e9a0d15954897317e6f5a0d80996239af3b376d5ddd527f73428ae
# Conflicts:
# contrib/devtools/symbol-check.py
# contrib/gitian-descriptors/gitian-linux.yml
* build: Add cmake as depends package (cmake.mk)
The bls-signatures library requires cmake 3.14
* depends: Update chia_bls to version 1.0.0 of dashpay/bls-signatures
* depends: Rename package chia_bls to bls-dash
* depends: Disable blspy/tests/benchmarks build for chia_bls
Note: Building with tests would require the following:
depends: Fix macOS build for versions < 10.12
Seems like older versions of macOS pretend to support c++17 std libs but do not have/have issues with uncaught_exceptions. "Catch", the testing framework used in the bls lib wants those by default but setting `DCATCH_CONFIG_NO_CPP17_UNCAUGHT_EXCEPTIONS` disables them which should be just fine to do in all cases here.
---
depends/packages/chia_bls.mk | 2 ++
1 file changed, 2 insertions(+)
diff --git a/depends/packages/chia_bls.mk b/depends/packages/chia_bls.mk
index e79a85844f..25749b74ab 100644
--- a/depends/packages/chia_bls.mk
+++ b/depends/packages/chia_bls.mk
@@ -28,6 +28,8 @@ define $(package)_set_vars
$(package)_config_opts_darwin+= -DCMAKE_AR="$(host_prefix)/native/bin/$($(package)_ar)"
$(package)_config_opts_darwin+= -DCMAKE_RANLIB="$(host_prefix)/native/bin/$($(package)_ranlib)"
endif
+
+ $(package)_cppflags+= -DCATCH_CONFIG_NO_CPP17_UNCAUGHT_EXCEPTIONS
endef
define $(package)_config_cmds
--
* depends: Drop obsolete variable
* bls: Integrate the upgraded version into the codebase
* depends: Pre-fetch relic to fix gitian
Can fetch apt packages only when building with lxc
Co-authored-by: UdjinM6 <UdjinM6@users.noreply.github.com>
4452829 gitian: quick hack to fix version string in releases (Cory Fields)
Pull request description:
Credit: @luke-jr
Release version strings were broken in Gitian by #7522. This is a minimal fix suitable for 0.15.
After this, we should fix up version handling for good so that gitian packages the correct string in the release tarball, so that git is not required to get the tag name.
Tree-SHA512: fa609a744c46306b0809f08fed6e96eff41b13e82f3e213711e4abef370558b64a68972f283a038330882cb6c40b32547fbb0f89b8058cc2c6025bff134473c3
* Remove use of -rdynamic
This causes check-symbols to fail horribly and also turned out to be not
required when using libbacktrace. It was only required when using
"backtrace()" from "<execinfo.h>"
* Remove spurious ], from configure.ac
* Add -DENABLE_STACKTRACES=1 to CMakeLists.txt
* Remove unused method my_backtrace_simple_callback
* Use fs::path().filename() instead of basename()
* Add static g_exeFileName and g_exeFileBaseName
* Use .exe.dbg file when available
* Use uint64_t instead of uintptr_t
* Implement GetBaseAddress() for unix and win32
* Implement unified crash_info and use it everywhere before printing crash info
* Print a serialized version of crash_info when there is no debug info
* Implement "-printcrashinfo" command line option
* Compile stacktrace support unconditionally and only make crash hooks conditional
This also renames the --enable-stacktraces option to --enable-crash-hooks
* Enable crash hooks in win/linux Gitian builds
* Try to load .debug file on MacOS and enable crash hooks for osx Gitian builds
* Check for dsymutil and if it needs --flat
* Create .debug files in osx Gitian build
* Handle review comments
* Also print crash description when no stacktrace is available
* Unconditionally add -g1 debug information
Instead of making it dependent on "--enable-crash-hooks". We will need the
debug info every time now, even in release builds.
* Put MacOS debug info into dSYM symbols instead of plain .debug files
* Implement MacOS specific GetBaseAddress
ed1fcdc Bugfix: Detect genbuild.sh in repo correctly (Luke Dashjr)
e98e3dd Bugfix: Only use git for build info if the repository is actually the right one (Luke Dashjr)
Tree-SHA512: 510d7ec8cfeff4e8e0c7ac53631eb32c7acaada7017e7f8cc2e6f60d86afe1cd131870582e01022f961c85a783a130bcb8fef971f8b110070c9c02afda020726
The cache can become quite large and uncompressing/recompressing it is slow.
Better to let ccache handle compression as it will then only perform it
on files which are actually used.
This fixes an issue reported by users on Ubuntu 18.04 LTS. They got the
following error message when starting dash-qt:
xkbcommon: ERROR: failed to add default include path auto
Qt: Failed to create XKB context!
Use QT_XKB_CONFIG_ROOT environmental variable to provide an additional search path, add ':' as separator to provide several search paths and/or make sure that XKB configuration data directory contains recent enough contents, to update please see http://cgit.freedesktop.org/xkeyboard-config/ .
Dash-qt starts then, but does not accept any input from the keyboard.
The reason is that qt tries to find the xkb config root in the configure
phase, but fails to find it. To make this detection work, libxkbcommon
has to be installed on the host system, even though it is then never used.
* Add stdin, stdout, stderr to ignored export list
* Merge #13171: Change gitian-descriptors to use bionic instead
c1afe3232fa10e290fb355cf37ea4c7bc1084065 Change gitian-descriptors to use bionic instead (Chun Kuan Lee)
Pull request description:
I have tested this with Ubuntu Bionic host with lxc 3.0.0, the gitian-build for Windows and MacOSX work fine, but there is an issue about it for Linux. Failed at check-symbol:
```
test/test_bitcoin: symbol __divmoddi4 from unsupported version GCC_7.0.0
test/test_bitcoin: symbol log2f from unsupported version GLIBC_2.27
qt/bitcoin-qt: symbol __divmoddi4 from unsupported version GCC_7.0.0
qt/bitcoin-qt: symbol log2f from unsupported version GLIBC_2.27
```
I think this should be fixed in `./configure --enable-glibc-back-compat`
Should not be merged before #13177devrandom/gitian-builder#178Close#12511
Tree-SHA512: 257d75d5b6864e105279f7a7b992fbbd7903cdbe3300b66dacec0a783d267707d9dbbfe0e64a36983ca1eca50a2a5e1cdb222b6d3745ccc3e5fc5636c88b581f
* Use IN6ADDR_ANY_INIT instead of in6addr_any
This is the same fix as fc6a9f2ab1
Couldn't backport the original commit as we are missing some refactorings.
* Use subdirectory for depends cache in gitian builds
* Make timestamps of wrappers deterministic
* Use ccache in gitian builds
* Upgrade ccache to latest version (3.4.2)
* Build the branch that belongs to the Jenkins build instead of develop
We mostly switched to python3, so we should also install it in gitian
builds. Especially the osx build needs as it otherwise fails due to
missing setuptools.
The -debug tarballs/zips contain detached debugging symbols. To use them, place
in the same dir as the target binary, and invoke gdb as usual.
Also, because the debug symbols add a substantial space requirement, the build
dirs are now deleted when they're no longer needed.
This removes the following executables from the binary gitian release:
- test_bitcoin-qt[.exe]
- bench_bitcoin[.exe]
@jonasschnelli and me discussed this on IRC a few days ago - unlike the
normal `bitcoin_tests` which is useful to see if it is safe to run
bitcoin on a certain OS/environment combination, there is no good reason
to include these. Better to leave them out to reduce the download
size.
Sizes from the 0.12 release:
```
2.4M bitcoin-0.12.0/bin/bench_bitcoin.exe
22M bitcoin-0.12.0/bin/test_bitcoin-qt.exe
```
This removes the following executables from the binary gitian release:
- test_bitcoin-qt[.exe]
- bench_bitcoin[.exe]
@jonasschnelli and me discussed this on IRC a few days ago - unlike the
normal `bitcoin_tests` which is useful to see if it is safe to run
bitcoin on a certain OS/environment combination, there is no good reason
to include these. Better to leave them out to reduce the download
size.
Sizes from the 0.12 release:
```
2.4M bitcoin-0.12.0/bin/bench_bitcoin.exe
22M bitcoin-0.12.0/bin/test_bitcoin-qt.exe
```
Github-Pull: #7776
Rebased-From: f063863d1f
