Commit Graph

1136 Commits

Author SHA1 Message Date
fanquake
6c46d02157 Merge #21672: doc: remove boostrap info from GUIX_COMMON_FLAGS doc
09b3e468480538977d2844ba083c66197c9d1d08 doc: remove boostrap info from GUIX_COMMON_FLAGS doc (fanquake)

Pull request description:

  Passing `ADDITIONAL_GUIX_COMMON_FLAGS="--no-substitutes --bootstrap"` as suggested doesn't work:
  ```bash
        ...outputting in: '/bitcoin/guix-build-a1f0b8b62eb8/output/x86_64-linux-gnu'
            ...bind-mounted in container to: '/outdir-base/x86_64-linux-gnu'
  guix time-machine: error: bootstrap: unrecognized option
  ```

  and I think bootstrapping is more than covered in the preceding "Choose your security model" section.

ACKs for top commit:
  dongcarl:
    ACK 09b3e468480538977d2844ba083c66197c9d1d08
  jarolrod:
    ACK 09b3e468480538977d2844ba083c66197c9d1d08
  laanwj:
    Documentation review ACK 09b3e468480538977d2844ba083c66197c9d1d08

Tree-SHA512: e533a0b925a2ec091884ec04313f56376b4e85f615c8853dd51840181493d61bd01fee1c019e34880c32d7896e4871ea77e63398ba02b022f614c03bb16531aa
2023-03-26 16:50:26 -05:00
fanquake
70154718f6 Merge #21661: doc: Fix name of script guix-build
08151e19d97f0407e99826007df3ba5b257129b4 doc: Fix name of script guix-build (Stephan Oeste)

Pull request description:

ACKs for top commit:
  hebasto:
    ACK 08151e19d97f0407e99826007df3ba5b257129b4
  jarolrod:
    ACK 08151e19d97f0407e99826007df3ba5b257129b4

Tree-SHA512: 50e52f91b489db6616b5c9a993474bc1b8c196c3cac4fd5ded7c8fece5a7d72f85d9f566ee6a3df56a132a22a91dd72801ce849ec5e430a7850ff05abcab6b37
2023-03-26 16:50:26 -05:00
W. J. van der Laan
0d5bc0b0f5 Merge #21304: guix: Add guix-clean script + establish gc-root for container profiles
867a5e172a23899a4a70eca4a396c64f1951745e guix: Register garbage collector root for containers (Carl Dong)
8f8b96fb542701b7717683caa3848390b24f77ab guix: Update hint messages to mention guix-clean (Carl Dong)
44f6d4f56b16e1dc5e8a23318b8e7aad0665f178 guix: Record precious directories and add guix-clean (Carl Dong)
84912d4b24382ae022da3a863bd6caa2b8948d94 build: Remove spaces from variable-printing rules (Carl Dong)

Pull request description:

  ```
  guix: Record precious directories and add guix-clean

  Many users have reported problems that stem from having an unclean
  working tree. To that end, I've written a guix-clean script which should
  help reset the working tree while respecting user-specified precious
  directories.

  Precious directories, such as:

  - SOURCES_PATH
  - BASE_CACHE
  - SDK_PATH
  - OUTDIR

  Should be preserved when cleaning the working tree, and are thus
  recorded in ./contrib/guix/var/precious_dirs.

  The ./contrib/guix/guix-clean script is able to parse that file and make
  sure to avoid them when cleaning out the working tree.
  ```

ACKs for top commit:
  laanwj:
    ACK 867a5e172a23899a4a70eca4a396c64f1951745e

Tree-SHA512: c498fad781ff5e6406639df2b91b687fc528273fdf266bcdba8f6eec3b3b37ecce544b6da0252f0b9c6717f9d88e844e4c7b72d1877bdbabfc6871ddd0172af5
2023-03-26 16:50:26 -05:00
fanquake
27032ca6be partial Merge #21626: doc: Fix typos from codespell
94c7dd9ac810a60f9c818c494273ad798cbac34b doc: Fix typos from codespell lint (Yerzhan Mazhkenov)

Pull request description:

  Typos from codespell linter: https://cirrus-ci.com/task/6677401661865984?logs=lint#L856
  - txrequest.cpp: `annoucements` ==> `announcements`
  - contrib/guix/README.md:298: `stil` ==> `still`
  - contrib/guix/guix-build:18: `invokable` ==> `invocable`
  - contrib/guix/libexec/prelude.bash:12: `invokable` ==> `invocable`
  - src/test/fuzz/tx_pool.cpp:37: `acess` ==> `access`
  - src/txorphanage.h:29: `orginating` ==> `originating`

ACKs for top commit:
  practicalswift:
    cr ACK 94c7dd9ac810a60f9c818c494273ad798cbac34b: thnaks fro fiixng tpyos!
  jarolrod:
    ACK 94c7dd9ac810a60f9c818c494273ad798cbac34b

Tree-SHA512: e0fac462a2f9e68b6a161c9f5d95b4d0648ce5c618fd7cd243d57db8f0256138b8823b166ea406b21e95586eae43047df1ef0df04616858082a39c1d1eb13a86
2023-03-26 16:50:26 -05:00
fanquake
e5f4614404 Merge #21611: Fix a typo in guix-build output
5c09bcadc4563abd2009e58e5528b130c6375565 Fix a typo in guix-build output (Pieter Wuille)

Pull request description:

  This was overlooked in #21375.

ACKs for top commit:
  fanquake:
    ACK 5c09bcadc4563abd2009e58e5528b130c6375565

Tree-SHA512: 81d8ad4061abb17d5f16ae72ab0c88df76d5a2f100cb9f471ca700d2e87583103036367ea1958a1066f9613f63908d1b2b35734a7eb77ec9850d4c8b079732e1
2023-03-26 16:50:26 -05:00
W. J. van der Laan
dfd1045284 Merge #21375: guix: Misc feedback-based fixes + hier restructuring
7476b46f1893a4858616d2a8456a7c43238851ed guix: Build dmg as a static binary (Carl Dong)
06d6cf6784421290e6235fe8684d5e08ed6f1b62 depends: libdmg-hfsplus: Skip CMake RPATH patching (Carl Dong)
65176ab5730dff34466caaecdd292625ef8294fc guix: Remove codesign_allocate+pagestuff from unsigned tarball (Carl Dong)
ca85679eb43b8375a95d82101977829d08fb1e1b guix: Use clang-toolchain instead of clang (Carl Dong)
1aec0eda8fd31a57b0621eea616398017c2ead98 guix: Fallback to local build for substitute-enabled Guix users (Carl Dong)
1742f8e12d163852df09575e03edcd3db73198ee guix: Add early health check for guix-daemon (Carl Dong)
c1ae726a13ecfa5e7e9fdc3030a8110b8bb263f8 guix: More thoroughly control native toolchain (Carl Dong)
39741128d3775d198dbee34dc827353bfd18acd8 guix: Supply --link-profile (Carl Dong)
d55a1056ee565afed64e42d6f6efb6b0adc5599b guix: Add troubleshooting documentation entries (Carl Dong)
7f401c953f8bb3574cec48561e13ef3b47dedc6e guix: Adapt guix-build to prelude, restructure hier (Carl Dong)
4eccf063b252bfe256cf72d363a24cf0183e926e guix: Remove guix-build.sh filename extension (Carl Dong)
7753357a7bae98ec775c707b9dec4cea1e945802 guix: Add source-able bash prelude and utils (Carl Dong)
e5b49a01f5d0f631e7f08f86ca8a2c2b8213319f guix: Create windeploy inside distsrc-* (Carl Dong)
3e9982ab3877eb8fe0a8c0cb3d847ac0913c7336 contrib: Silence git-describe when looking for tag (Carl Dong)
d5a71e97853ea9e1b879e8c76bfb01d4bef33172 guix: Use --cores instead of --max-jobs (Carl Dong)

Pull request description:

  This PR addresses a few hiccups encountered by the brave souls who've been experimenting with the Guix scripts:
  - Resolves confusion between `--cores=` and `--max-jobs=`
    - `guix`'s `--cores=` actually corresponds to make's `--jobs=`, so let's just control `--cores=` with our overridable env var
  - `git-describe` will scream `fatal: no tag exactly matches '<hash>'` when looking for a tag, but we don't care, so silence that
  - `windeploy/unsigned` should be inside `distsrc-*` and created idempotently (sorry I know this one annoyed people)
  - Add troubleshooting documentation to `README.md`
  - Add early health check for `guix-daemon` in case user forgot to start a `guix-daemon`
  - Depending on configuration, a `--fallback` flag may be needed to tell Guix to not fail if substitutes fail but fallback to building locally
  - `codesign_allocate` and `pagestuff` are now unnecessary for codesigning as we're now using `signapple`

  A few robustness changes are also included:
  - We supply the `--link-profile` flag, as some Guix packages may expect the profile to be available under `$HOME/.guix-profile`
  - We now clear and manually set all toolchain-related env vars (e.g. `C*_INCLUDE_PATH`) ourselves, after patching a Qt::moc bug
  - We use the native `clang-toolchain` package for darwin builds instead of `clang`, lining up with all our other toolchain packages.

  Finally, we restructure the guix building hierarchy such that it looks something like:
  ```
  guix-build-<short-hash-or-version-tag>
  ├── distsrc-<short-hash-or-version-tag>-${HOST}
  │   ├── contrib
  │   ├── depends
  │   ├── src
  │   └── ...
  ├── distsrc-<short-hash-or-version-tag>-...
  └── output
      ├── dist-archive
      │   └── bitcoin-<short-hash-or-version-tag>.tar.gz
      ├── *-linux-*
      │   ├── bitcoin-<short-hash-or-version-tag>-*-linux-*-debug.tar.gz
      │   └── bitcoin-<short-hash-or-version-tag>-*-linux-*.tar.gz
      ├── x86_64-apple-darwin18
      │   ├── bitcoin-<short-hash-or-version-tag>-osx64.tar.gz
      │   ├── bitcoin-<short-hash-or-version-tag>-osx-unsigned.dmg
      │   └── bitcoin-<short-hash-or-version-tag>-osx-unsigned.tar.gz
      └── x86_64-w64-mingw32
          ├── bitcoin-<short-hash-or-version-tag>-win64-debug.zip
          ├── bitcoin-<short-hash-or-version-tag>-win64-setup-unsigned.exe
          ├── bitcoin-<short-hash-or-version-tag>-win64.zip
          └── bitcoin-<short-hash-or-version-tag>-win-unsigned.tar.gz
  ```
  Separating guix builds by their version identifier (basically namespacing them) allows us to change the layout in the future without worry about potential naming conflicts.

ACKs for top commit:
  sipa:
    ACK 7476b46f1893a4858616d2a8456a7c43238851ed
  laanwj:
    ACK 7476b46f1893a4858616d2a8456a7c43238851ed

Tree-SHA512: 0e899aa941aafdf552b2a7e8a08131ee9283180bbef7334439e2461a02aa7235ab7b9ca9c149b80fc5d0a9f4bbd35bc80fcee26197c0836ba8eaf2d86ffa0386
2023-03-26 16:50:26 -05:00
Wladimir J. van der Laan
5fda1ea15a Merge #21337: guix: Update darwin native packages dependencies
c967fb7fb97a86fc434912d0fff621f9f50dfe80 guix: Remove libcap from manifest (Hennadii Stepanov)
7bbb409314c63750a9731dedaaf6dd6ecd17d988 guix: Update darwin native packages dependencies (Hennadii Stepanov)

Pull request description:

  It is a #20470 follow up.

ACKs for top commit:
  fanquake:
    ACK c967fb7fb97a86fc434912d0fff621f9f50dfe80

Tree-SHA512: 66ce05770f578ba61a44c58747c5a2669f425a989ed987838058bd86e3b49e342ac5a4f8852fc49f2b3a86b58fb6a340fdf3e34c1fc19bdab910729febba4bc7
2023-03-26 16:50:26 -05:00
fanquake
2c4bf310df partial Merge #21325: lint: Fix spelling errors in comments
fbbb2d4fc13971c98c83a51635166ac532e71a32 lint: Fix spelling errors in comments (fyquah)

Pull request description:

  Found some spelling errors while running spelling linter  https://github.com/bitcoin/bitcoin/pull/21245

  This PR fixes them.

ACKs for top commit:
  fanquake:
    ACK fbbb2d4fc13971c98c83a51635166ac532e71a32 - I thought we just fixed all of these.

Tree-SHA512: 95525040001f94e899b778c616cb66ebafb679dff88835b66fccf6349d8eb942d6b7374c536a44e393f13156bce9a32ed57e6a82bb02074d2b3cddb2696addb2
2023-03-26 16:50:26 -05:00
fanquake
af614fece0 Merge #21298: guix: Bump time-machine, glibc, and linux-headers
c33b199456e57d83c21eacd36d3c56d0a123b0d0 guix: Bump glibc and linux-headers (Carl Dong)
65363a1bd8b886f5aef5fbc97ca88c9c9b243b21 guix: Rebase on 95aca2991b (1.2.0-12.dffc918) (Carl Dong)

Pull request description:

  On bumping the time-machine:

  ```
  A few changes which are useful for us:

  1. 'gnu: cross-gcc-arguments: Enable 128 bit long double for POWER9.' is
     now merged into master.
  2. gnutls is bumped to 3.6.15 and the temporal test failure in
     status-request-revoked is fixed. Note that this does not fix the case
     where one has installed Guix v1.2.0 and is running a substitute-less
     bootstrap build, since the `guix time-machine` command itself has a
     dependency on gnutls v3.6.12 (the one with the broken test) and will
     thus try to build it before attempting to jump forwards in time. This
     does however, mean that those who build a version of Guix that also
     contains this fix will not go backwards in time to build the broken
     gnutls v3.6.12.
  ```

  On bumping the rest:

  ```
  Bump glibc and linux-headers to match those of our Gitian counterparts.

  We also require a glibc >= 2.28 for the test-symbol-check scripts to
  work properly.

  The default BASE-GCC-FOR-LIBC also has to be bumped since glibc 2.31
  requires a gcc >= 6.2
  ```

  This is a prerequisite for #20980

ACKs for top commit:
  fanquake:
    ACK c33b199456e57d83c21eacd36d3c56d0a123b0d0 - I think going ahead with this now and to sycn back up to gitian is fine. It will also unblock #20980. Potential code signing related issues can be sorted out in #21239 and later PRs.

Tree-SHA512: 31f022aadb93ba44813b0da005b1f2e5d67d76e8cdcdb53368924d1ea6cb076a21218c26831a6b0dcdcfe33507f54934330489ba557371d740f5587b7d727b95
2023-03-26 16:50:26 -05:00
fanquake
86a6cf3005 Merge #21323: guix, doc: Update default HOSTS value
a0a7a4337d06553ab625bbd66ed4198c4bf7f18c guix, doc: Update default HOSTS value (Hennadii Stepanov)

Pull request description:

  This is a #21089 follow up.

ACKs for top commit:
  fanquake:
    ACK a0a7a4337d06553ab625bbd66ed4198c4bf7f18c

Tree-SHA512: c1813cc2b9212a79fd34d4e25cd0816b58264e1890daf777cd59411bd20fcc9affe312871d06fab1308b8f55c1a78ac1101e631882c18360a4709ecef4529f05
2023-03-26 16:50:26 -05:00
Wladimir J. van der Laan
42f4e752dd Merge #21321: guix: Add curl to required tool list
97f10621f4b76fbfcc0849b10d5edb4d6898ec5e guix: Add curl to required tool list (Hennadii Stepanov)

Pull request description:

  On Ubuntu Hirsute (minimum installation) with the system `guix` package:
  ```
  $ HOSTS=x86_64-linux-gnu ./contrib/guix/guix-build.sh
  make: Entering directory '/home/hebasto/bitcoin/depends'
  make[1]: Entering directory '/home/hebasto/bitcoin/depends'
  Checksum missing or mismatched for boost source. Forcing re-download.
  Fetching boost_1_71_0.tar.bz2 from https://dl.bintray.com/boostorg/release/1.71.0/source/
  /bin/sh: 1: curl: not found
  Fetching boost_1_71_0.tar.bz2 from https://bitcoincore.org/depends-sources
  /bin/sh: 1: curl: not found
  make[1]: *** [funcs.mk:276: /home/hebasto/bitcoin/depends/sources/download-stamps/.stamp_fetched-boost-boost_1_71_0.tar.bz2.hash] Error 127
  make[1]: Leaving directory '/home/hebasto/bitcoin/depends'
  make: *** [Makefile:281: download-linux] Error 2
  make: Leaving directory '/home/hebasto/bitcoin/depends'
  ```

  This PR fixes that issue.

ACKs for top commit:
  laanwj:
    ACK 97f10621f4b76fbfcc0849b10d5edb4d6898ec5e
  jonasschnelli:
    utACK 97f10621f4b76fbfcc0849b10d5edb4d6898ec5e

Tree-SHA512: b9e1ba966782b13e25c85d4a9cfd381bcda190244cb9821ae1a72fd51f82f23ed5e98d9a02384619e3299feb7a34e1da8b9190081a6005221f47f93ae18f3c0b
2023-03-26 16:50:26 -05:00
Wladimir J. van der Laan
a305fc4c51 Merge #21272: guix: Passthrough SDK_PATH into container
13a9fd11a507fd3398bc2c0a0575bdc81579243f guix: Passthrough SDK_PATH into container (Carl Dong)

Pull request description:

  This is a usability improvement for Guix builders so that they don't have to extract the Xcode tarball into `depends/SDKs` every time.

  Inspiration: https://github.com/bitcoin/bitcoin/pull/21089#issuecomment-778639698

ACKs for top commit:
  laanwj:
    Tested ACK 13a9fd11a507fd3398bc2c0a0575bdc81579243f

Tree-SHA512: 63392d537e48a0da9f0ee04a929613b139bef1ac5643187871c9ea5376afd2a3d95df0f5e0950ae0eccd2813b166667be98401e5a248ae9c187fe4e84e54d427
2023-03-26 16:50:26 -05:00
fanquake
0b78348aa1 Merge #21271: guix: Explicitly set umask in build container
d98f4593cf00ab2973f8113e30506861b24383bc guix: Explicitly set umask in build container (Carl Dong)

Pull request description:

  Opened as a separate PR to fix non-reproducibility found through testing here: https://github.com/bitcoin/bitcoin/pull/21089#issuecomment-783549633

  Many thanks to everyone who helped find this!

ACKs for top commit:
  laanwj:
    ACK d98f4593cf00ab2973f8113e30506861b24383bc
  fanquake:
    ACK d98f4593cf00ab2973f8113e30506861b24383bc - I'm seeing matching hashes.

Tree-SHA512: ea339c3902f2f4dea32e8ef5cc675a1df0679530881260ae999aaaf7339d5b12c46e01e58677cbb079f33e573ad105e2b443a835f3e944ef8e943a25f83027f1
2023-03-26 16:50:26 -05:00
fanquake
12a586e35a Merge #21078: guix: only download sources for hosts being built
a6a1b106dcc4350e420c461171c47e4934087175 guix: only download sources for hosts being built (fanquake)

Pull request description:

  For example, if a user is only interested in building for Linux, this saves downloading the macOS compiler and additional dependencies, which is meaningful on a slow/poor connection. This will result in a few additional `make` invocations, for the Linux hosts, however this is low overhead, and time-wise irrelevant in terms of the overall build.

ACKs for top commit:
  laanwj:
    Code review ACK a6a1b106dcc4350e420c461171c47e4934087175

Tree-SHA512: 34c916ae6f69fed0d5845690b39111a8bee37208fd727176f375cf5eb4860f512abe12bde2680d697c859b4d50a3bc5688ddca7c2f28f9968fcf358753cf3f6d
2023-03-26 16:50:26 -05:00
fanquake
9c8f5f71dc partial Merge #21089: guix: Add support for powerpc64{,le}
95990b9f3278360b63e79d6975af4ab5009c66ba guix: Update conservative space requirements (Carl Dong)
5e6df1132656995ce5b9ce279d5a9808ea52ab32 guix: Add support for powerpc64{,le} (Carl Dong)

Pull request description:

  ```
  The new time-machine commit contains a few small changes that make the
  powerpc cross-toolchain work.
  ```

  See this compare to review my custom patches to Guix: 7d6bd44da5...6c9d16db96

ACKs for top commit:
  fanquake:
    ACK 95990b9f3278360b63e79d6975af4ab5009c66ba

Tree-SHA512: 464b0fb93d65962d8c27499293edb618d13d18f40d44e3eed96935e86d430666dfb1c5b8a30f99ffdfd17b44514ad88e358977390b689a2e3831d521f6f7b86a
2023-03-26 16:50:26 -05:00
Wladimir J. van der Laan
cd0d7d2991 Merge #21088: guix: Jump forwards in time-machine and adapt
d02076b8852d8faae95cee6e3de434460c07412a guix: Jump forwards in time-machine and adapt (Carl Dong)
f8ca8c5c28d3050b780e67d47a50ac65fc2dc3ad guix: Supply --keep-failed for debugging (Carl Dong)

Pull request description:

  ```
  The new time-machine commit is Guix v1.2.0 with a yet-unupstreamed patch
  for NSIS.

  A few important changes:

  1. Guix switched back from using CPATH to C{,PLUS}_INCLUDE_PATH as the
     way to indicate #include search paths.
  2. GCC's library is now split into a separate output, whereas before it
     was included in the default output. This means that our gcc toolchain
     packages need to propagate that output.
  3. A few package versions were bumped
  ```

  See this compare to review my custom patches to Guix: https://github.com/dongcarl/guix/compare/version-1.2.0...7d6bd44da57926e0d4af25eba723a61c82beef98

ACKs for top commit:
  laanwj:
    ACK d02076b8852d8faae95cee6e3de434460c07412a

Tree-SHA512: 896d5bf1b6e5fda2f0106013c568c119bbbb86cb31a8c0a22432bada9b7da51678b96374bf8fd7c15353698ba47ac9dd39874d40c39001281471db7c78bf1705
2023-03-26 16:50:26 -05:00
fanquake
2f3b44289c Merge #21087: guix: Passthrough BASE_CACHE into container
901f54321b386258a1682423160bfdfa35ea4c39 guix: Passthrough BASE_CACHE into container (Carl Dong)

Pull request description:

  This allows depends-built packages to be cached.

ACKs for top commit:
  MarcoFalke:
    Approach ACK 901f54321b386258a1682423160bfdfa35ea4c39
  fanquake:
    ACK 901f54321b386258a1682423160bfdfa35ea4c39

Tree-SHA512: 464815f41fc081d7956bec84380668834b6ee6751c7a3d56daad6e1fc91e582de4bbdd1a89f399b1136f2adc4d9941517cfe4db694f0ee5bf59bf2f44fc6fda0
2023-03-26 16:50:26 -05:00
Wladimir J. van der Laan
0cce5d13b6 Merge #21163: doc: Guix is shipped in Debian and Ubuntu
fa051c23860bcdcc871db5ad6b51b8d9ca88da35 doc: Guix is shipped in Debian and Ubuntu (MarcoFalke)

Pull request description:

ACKs for top commit:
  practicalswift:
    ACK fa051c23860bcdcc871db5ad6b51b8d9ca88da35 🚀

Tree-SHA512: f72f546cfc20cf1cc0c26c2306ac06416ada87661596fe811b497cce646aa286dc4aee832145bf838b13fbd3c5f064519eb8c0b4525eb562f2f04f20e2876ffc
2023-03-26 16:50:26 -05:00
fanquake
89ff5e3ef8 Merge #21116: build: Disable --disable-fuzz-binary for gitian/guix builds
cb151b797aeabb376d37ef058aee34c06d36d487 build: Disable --disable-fuzz-binary for guix builds (Hennadii Stepanov)
fd7caae35fd3f83175247a79f7573e374779a851 build: Disable --disable-fuzz-binary for gitian builds (Hennadii Stepanov)

Pull request description:

  Fuzz binary is not shipped to users.
  This PR saves hundreds MB of the disk space for containers.

ACKs for top commit:
  MarcoFalke:
    review ACK cb151b797aeabb376d37ef058aee34c06d36d487
  fanquake:
    ACK cb151b797aeabb376d37ef058aee34c06d36d487

Tree-SHA512: 858e3816576c307b47915bb05de79a28029beaef8835c01f1bd6a764a0cf7f7f63ef8c2dc2c5944cb36cc9f4788d9b0590b8a5dda96940167252ba371cdbd078
2023-03-26 16:50:26 -05:00
fanquake
a6e7dda55b Merge #17920: guix: Build support for macOS
f1694757ddbcb3635213b085e864851e285c8c12 guix: Fix typo (Carl Dong)
771c4b98a8693eee642f2b118b3193fe6e022291 guix: README: Add darwin HOSTS entry (Carl Dong)
8dbf18cb1d3260d34ba822ceb12e67b1f124ea13 guix: Check for macOS SDK before building anything (Carl Dong)
34b23f597ec52efb795d72e9e5620712d0010edd guix: Set ZERO_AR_DATE for darwin build determinism (Carl Dong)
f3835dc6a3732dcd4afbb5987f84dc27f2bf55af build: Make xorrisofs reproducible with -volume_date (Carl Dong)
c9eb4cf3a0f81bfd72f06fd43b5610f0a4f5e804 guix: Add support for darwin builds (Carl Dong)
37fe73a092b08fe9d7ce636a1021429de6cda757 build: Add var printing target to src/Makefile.am (Carl Dong)

Pull request description:

  This PR brings our Guix builds on par with Gitian in terms of supported architectures.

  Reviewers: if you run a build, please submit:

  ```
  find output/ -type f -print0 | env LC_ALL=C sort -z | xargs -r0 sha256sum
  ```

  So that we can compare hashes and ensure reproducibility!

ACKs for top commit:
  fanquake:
    ACK f1694757ddbcb3635213b085e864851e285c8c12 - I think we can make some small usability improvements, but this is ok to merge now.

Tree-SHA512: 4af2b71654a9736467dcc681d10601c6eee37800d7847011a50585455b67b55d61742ca5604585f310a2fd75335b674e5e27dfb5169cb2f26e112aa4c411d8be
2023-03-26 16:50:26 -05:00
fanquake
1c79710e65 Merge #20937: guix: Make nsis reproducible by respecting SOURCE-DATE-EPOCH
1fca9811e1331ac5dae8188f6178cc37da4929a7 lint: Skip whitespace lint for guix patches (Carl Dong)
a91c46c57d88fc399432afab7bb0fb14c3e490a7 guix: Make nsis reproducible by respecting SOURCE-DATE-EPOCH (Carl Dong)

Pull request description:

  ```
  When building nsis, if VERSION is not specified, it defaults to
  cvs_version which is non-deterministic as it includes the current date.

  This patches nsis to default to SOURCE_DATE_EPOCH if it exists so that
  nsis is reproducible.

  Upstream change: https://github.com/kichik/nsis/pull/13
  ```

  Sidenote: also a good demonstration of how Guix allows us to flexibly patch our tools!

  Note to reviewers: if you want to compare hashes, please build after Jan 16th 2021 without my substitute server enabled!

ACKs for top commit:
  fanquake:
    ACK 1fca9811e1331ac5dae8188f6178cc37da4929a7

Tree-SHA512: b800e0ce5f73827ad353739effb9167ec3a6bdb362c725ae20dd3f025ce78660f85c70ce1d75cd0896facf1e8fe38a9e058459ed13dec71ab3a2fe41e20eaa5d
2023-03-26 16:50:26 -05:00
fanquake
d4d19b539b Merge #20619: guix: Quality of life improvements
570e43fe72e13e0a82e25f7145704f62b2c2cc52 guix: Print build params inside/outside of container (Carl Dong)
2f9d1fdde66f4713351905ec73487e5288d20f8f guix: Move DISTSRC determination to guix-build.sh (Carl Dong)
0b7cd07bb56baa112ffa596fb23a905871031a36 guix: Move OUTDIR determination+creation to guix-build.sh (Carl Dong)
d27ff8b86aa66acec63b5713912bd4ad9470e66f guix: Add more sanity checks to guix-build.sh (Carl Dong)
57f95331464f097261c63fd1b6040536c58a03fa guix: Add section headings to guix-build.sh (Carl Dong)
38b7b2ed72b1f0f57bd9800c7fbb7b7c98a20ed0 genbuild: Specify rev-parse length (Carl Dong)
036dc740da3239cdcc13e0f299ab95b456f7118b docs: Point to contrib/guix/README.md in doc/guix.md (Carl Dong)
34f0fda2d31d2ada632ca1165b82aebdfd342efe guix: Small updates to README wording (Carl Dong)
402e3a5b1ed9de7057ce9955ea792ad1c2b9f2b5 guix: Update HOSTS README entry for new architectures (Carl Dong)
cfa7ceb21b14d1fa24c2541bf242a0ed539b9e1b guix: Remove README development environment section (Carl Dong)
93b6a8544a03d13733ca2ef769f76df587ad86c8 guix: Add ADDITIONAL_GUIX_{COMMON,TIMEMACHINE}_FLAGS options (Carl Dong)
0f31e24703e25698d2d41fb54e30ec75a4a80943 guix: Add SUBSTITUTE_URLS option (Carl Dong)
444fcfca907d46cfeb52001599966cce25bdf54e guix: Make guix honor MAX_JOBS setting (Carl Dong)

Pull request description:

  After live-demo-ing a Guix build (which completed successfully!) on achow101's stream, I realized there were a few quality of life improvements which can be made to improve the user experience of our Guix build process. Here are a few of them.

  Notable changes:
  1. When `MAX_JOBS` is specified, both `guix time-machine` and `guix environment` will now build up to `MAX_JOBS` packages at a time when creating the build environment
  2. The instructions for using substitutes were incorrect, and has now been replaced with a `SUBSTITUTE_URLS` environment variable, which works well with shell's IFS splitting rules
  3. New `ADDITIONAL_GUIX_{COMMON,TIMEMACHINE}_FLAGS` options, for more granular customization of the build process.
  4. README cleanup

ACKs for top commit:
  fanquake:
    ACK 570e43fe72e13e0a82e25f7145704f62b2c2cc52 - lets move this forward.

Tree-SHA512: 4e8ab560522ade5efb5e8736aec0fb1a3f19ae9deb586c1ab87020816876f3f466a950b3f8c04d9fa1d072ae5ee780038c5c9063577049bdd9db17978e11c328
2023-03-26 16:50:26 -05:00
Wladimir J. van der Laan
de430e69aa Merge #20318: build: Ensure source tarball has leading directory name
faa2f06f5eaf8578873495f44603ee74d7a1abf4 scripted-diff: [build] Ensure source tarball has leading directory name (MarcoFalke)

Pull request description:

  This has been fixed in 0.20, so it needs to be fixed on master as well to avoid a regression

  #18945

ACKs for top commit:
  laanwj:
    ACK faa2f06f5eaf8578873495f44603ee74d7a1abf4
  hebasto:
    ACK faa2f06f5eaf8578873495f44603ee74d7a1abf4, tested gitian builds only.
  promag:
    ACK faa2f06f5eaf8578873495f44603ee74d7a1abf4.

Tree-SHA512: e3b025c29c45b025002abc35262bb5d771f6cbd807f1c256c477c243685e93cd43ad9f642b38e3cf218590912abe6ea0ddfec3bfbef36f99080aad74ed6cc0af
2023-03-26 16:50:26 -05:00
fanquake
a3ceaea97e Merge #18958: guix: Make V=1 more powerful for debugging
f852761aec81ed23c7b9e4546c08d1ef303f2507 guix: Add clarifying documentation for V env var (Carl Dong)
85f4a4b0822e3aa10310c4623eff719f301e9263 guix: Make V=1 more powerful for debugging (Carl Dong)

Pull request description:

  ```
  - Print commands in both unexpanded and expanded forms
  - Set VERBOSE=1 for CMake
  ```

  Ping MarcoFalke hopefully you use `V=1` already for the Guix builds on DrahtBot?

ACKs for top commit:
  fanquake:
    ACK f852761aec81ed23c7b9e4546c08d1ef303f2507. Ran a Windows Guix build and compared the output from master and this PR when using `V=1`. i.e `HOSTS=x86_64-w64-mingw32 PATH="/root/.config/guix/current/bin${PATH:+:}$PATH" V=1 ./contrib/guix/guix-build.sh`.

Tree-SHA512: 8bc466fa7b869618bbd5a0a91c6b23d4785009289f8dfb93b0349317463a9ab9ece128c72436e02a0819722a63e703100aed15807867a716fda891292fcb9d9d
2023-03-26 16:50:26 -05:00
PastaPastaPasta
bc20ef6bf5
ci: resolve gitian automation failures on dashevo repo (#5267)
## Issue being fixed or feature implemented
Automation in private dashevo GitHub was failing due to the branch
trying to be built not having the automation be the newer tagged based
system. as such, if you simply tag a commit from the dashpay repo and
try to build it using automation, the build would fail. This should
resolve the issue. Recommended to BP so that we don't have this issue
when building v19.x branch builds

## What was done?


## How Has This Been Tested?
Creating a docker image should now be as simple pushing a branch and tag
to dashevo/dash; start the automation on branch develop with tag
previously specified

## Breaking Changes


## Checklist:
- [x] I have performed a self-review of my own code
- [ ] I have commented my code, particularly in hard-to-understand areas
- [ ] I have added or updated relevant unit/integration/functional/e2e
tests
- [ ] I have made corresponding changes to the documentation

**For repository code-owners and collaborators only**
- [x] I have assigned this pull request to a milestone
2023-03-23 09:51:05 -05:00
Konstantin Akimov
e3a211a8a4
chrore: bump version in gitian to 19 (#5220)
Name of some files in gitian's build contains "18" instead 19.

## Issue being fixed or feature implemented


## What was done?
Version bumped in gitian's related configs

## How Has This Been Tested?
Will be tested with next RC of v19

## Breaking Changes
No breaking changes


## Checklist:
- [x] I have performed a self-review of my own code
- [ ] I have made corresponding changes to the documentation
@PastaPastaPasta may you update release TODO checklist, please?
- [x] I have assigned this pull request to a milestone
2023-02-27 23:11:07 -06:00
fanquake
700d46b26d Merge #18741: guix: Make source tarball using git-archive
bfe1ba2f5b36056e0c41edf8206b93d3d83098df rel-builds: Specify core.abbrev for git-rev-parse (Carl Dong)
27e63e01cce368d67092de8f0c736927d6f6aa69 build: Accomodate makensis v2.x (Carl Dong)
1f2c39a30e0f82046c7aecddfda3eb99cb536816 guix: Remove logical cores requirement (Carl Dong)
a4f6ffa71e335d4b2a6bf525b7f416968f9cd9f7 lint: Also enable source statements for non-gitian (Carl Dong)
d256f91cb1b0d6ff5170106b99b0266cbe51f5a2 rel-builds: Directly deploy win installer to OUTDIR (Carl Dong)
fa791da02f9684e3fd554b687fb692ae6a23d65a nsis: Specify OutFile path only once (Carl Dong)
14701604d0904bc5bbf1c67de08f8ee6d3215523 guix: Expose GIT_COMMON_DIR in container as readonly (Carl Dong)
f5a6ac4f48b18f93050d77bcb23f9cf45ec34647 guix: Make source tarball using git-archive (Carl Dong)
395c1137f630dc495ffb2752a23bc1dfd470ee53 gitian: Limit sourced script to just assignments (Carl Dong)

Pull request description:

  Based on: #18556
  Related: https://github.com/bitcoin/bitcoin/pull/17595#discussion_r399728721

ACKs for top commit:
  fanquake:
    ACK bfe1ba2f5b36056e0c41edf8206b93d3d83098df - I agree with Carl, and am going to merge this. I'd like for Linux Guix builds to be working again, and we can rebase #18818.

Tree-SHA512: c87ada7e3de17ca0b692a91029b86573442ded5780fc081c214773f6b374a0cdbeaf6f6898c36669c2e247ee32aa7f82defb1180f8decac52c65f0c140f18674
2023-02-20 09:09:23 -06:00
fanquake
87de9c8f5c Merge #17929: build: add linker optimisation flags to gitian & guix (Linux)
f2b5b0a3b48f9241971c14238040048f8b630811 build: add linker optimization flags to guix (fanquake)
b8b050a8d642e38c81d1e451750c2c8db92fee5e build: add linker optimization flags to gitian descriptors (fanquake)

Pull request description:

  This PR adds `-Wl,O2` to our gitian and guix LDFLAGS. This makes the linker perform certain optimisations (and is different from LTO).

  Any -O argument will enable optimizations in GNU ld. We can use -O2 here, as this matches our compile flags. Note that this would also enable additional optimizations if using the lld or gold linkers, when compared to -O0.

  A nice writeup + diagrams of some of these optimizations is  available here: http://lwn.net/Articles/192624/.

  #### master
  ```bash
  # bitcoind
  Histogram for `.gnu.hash' bucket list length (total of 3 buckets)
   Length  Number     % of total  Coverage
        0  1          ( 33.3%)       0.0%
        1  0          (  0.0%)       0.0%
        2  1          ( 33.3%)      40.0%
        3  1          ( 33.3%)     100.0%
  ```
  ```bash
  # bitcoin-qt
  Histogram for `.gnu.hash' bucket list length (total of 3 buckets)
   Length  Number     % of total  Coverage
        0  0          (  0.0%)       0.0%
        1  1          ( 33.3%)      10.0%
        2  0          (  0.0%)      10.0%
        3  0          (  0.0%)      10.0%
        4  1          ( 33.3%)      50.0%
        5  1          ( 33.3%)     100.0%
  ```

  #### this PR:
  ```bash
  # bitcoind
  Histogram for `.gnu.hash' bucket list length (total of 8 buckets)
   Length  Number     % of total  Coverage
        0  3          ( 37.5%)       0.0%
        1  5          ( 62.5%)     100.0%
  ```
  ```bash
  # bitcoin-qt
  Histogram for `.gnu.hash' bucket list length (total of 19 buckets)
   Length  Number     % of total  Coverage
        0  9          ( 47.4%)       0.0%
        1  10         ( 52.6%)     100.0%
  ```

  #### GNU ld -O

  > If level is a numeric values greater than zero ld optimizes the output. This might take significantly longer and therefore probably should only be enabled for the final binary. At the moment this option only affects ELF shared library generation. Future releases of the linker may make more use of this option. Also currently there is no difference in the linker’s behaviour for different non-zero values of this option. Again this may change with future releases.

  #### lld -O

  > Optimize output file size

ACKs for top commit:
  dongcarl:
    ACK f2b5b0a3b48f9241971c14238040048f8b630811
  laanwj:
    ACK f2b5b0a3b48f9241971c14238040048f8b630811

Tree-SHA512: e53f3a4338317dbec65d3a93b57b5a6204aabdf9ac82d99447847a3c8627facc53c58c2cf947376f13edd979fc8129a80f18d9ebeccd191a576c83f1dad5c513
2023-02-20 09:09:23 -06:00
fanquake
dff599acff Merge #17595: guix: Enable building for x86_64-w64-mingw32 target
a35e3235891d35daa167116cc70340140e883f06 guix: Appease travis. (Carl Dong)
0b66d22da5f53640e22f05adf880782c613e6d0f guix: Use gcc-9 for mingw-w64 instead of 8 (Carl Dong)
ba0b99bdd613ba7f17c6247ece3001e1b44759a3 guix: Don't set MINGW_HAS_SECURE_API CFLAG in depends (Carl Dong)
93439a71eda49fb69f1e82966a23a946733aa6fa guix: Bump to upstream commit with mingw-w64 changes (Carl Dong)
35a96792dda9e78165b1598aeac7b2ab759e7be5 guix: Check mingw symbols, improve SSP fix docs (Carl Dong)
449d8fe25bbe25daacfc67aa89ca32b0a3254c5a guix: Expand on INT trap message (Carl Dong)
3f1f03c67a8e9edf487f08d272adb18b0a3942c8 guix: Spelling fixes (Carl Dong)
ff821dd2a1c600488d11e7d9a20e9179ecc9144b guix: Reinstate make-ssp-fixed-gcc (Carl Dong)
360a9e0ad50a36ec79a1a160dbed3966689fd41c guix: Bump time-machine for mingw-w64 patches (Carl Dong)
93e41b7e3b54c17fd1b4c61ee95fc0dc2827e954 guix: Use gcc-8 for mingw-w64 instead of 7 (Carl Dong)
ef4f7e4c45c60a69406134122f091c77c6ef740f guix: Set the well-known timezone env var (Carl Dong)
acf4b3b3b5accf60a19441a0298ef27001b78e72 guix: Make x86_64-w64-mingw32 builds reproducible (Carl Dong)
c4cce00eac691625b78b92f7dba0b7f57def19e5 guix: Remove dead links from README. (Carl Dong)
df953a4c9a6143f45864757b706c88b6fa70545a guix: Appease shellcheck. (Carl Dong)
91897c95e191d293eb27d8af15cbeafc5b8f3895 guix: Improve guix-build.sh documentation (Carl Dong)
570d769c6c59b9f6d1a2b95b2ed60432cb33b3ba guix: Build support for Windows (Carl Dong)

Pull request description:

  ~~Based on: https://github.com/bitcoin/bitcoin/pull/16519~~
  Based on: #17933 (Time Machines are... shall we say... superior 😁)

  This PR allows us to perform Guix builds for the `x86_64-w64-mingw32` target. We do this _without_ splitting up the build script like we do in Gitian by using this newfangled alien technology called `case` statements. (This is WIP and might be changed to `if` statements soon)

ACKs for top commit:
  fanquake:
    ACK a35e3235891d35daa167116cc70340140e883f06 2/3

Tree-SHA512: c471951c23eb2cda919a71285d8b8f2580cb20f09d5db17b53e13dbd8813e01b3e7a83ea848e4913fd0f2bc12c6c133c5f76b54e65c0d89fed4dfd2e0be19875
2023-02-20 09:09:23 -06:00
fanquake
355a735e78 Merge #18320: guix: Remove now-unnecessary gcc make flag
0ae42a16c766a7ecb8711bfad6f22b8581ea0258 guix: Remove now-unnecessary gcc make flag (Carl Dong)

Pull request description:

  ```
  Previously, Guix would produce a gcc which did not know to use the SSP
  function from glibc, and required a gcc make flag for it to do so, in my
  attempt to fix it upstream I realized that this is no longer the case.

  This can be verified by performing a Guix build and doing

    readelf -s ... | grep __stack_chk

  to check that symbols are coming from glibc, and doing

    readelf -d ... | grep NEEDED | grep ssp

  to see that libssp.so is not being depended on
  ```

ACKs for top commit:
  fanquake:
    ACK 0ae42a16c766a7ecb8711bfad6f22b8581ea0258 - ran a Guix build (hashes below) and checked all the linux binaries:

Tree-SHA512: 701b91e7c323b12a29af9539cb2656d10ce0a93af573a02e57f0b7fea05a6e1819798536eadb24d0a17e7f35b503f5e863fee5e7409db1b8a3973c4375e49d4e
2023-02-20 09:09:23 -06:00
fanquake
1f9f160d24 Merge #18104: build: Skip i686 build by default in guix and gitian
fae9084ac5b10f94bdee54853d307838c4254e9c build: Skip i686 build by default in guix and gitian (MarcoFalke)
fa55a2554c2661b8f2a759044d5ac85c9979d9ca depends: Remove reference to win32 (MarcoFalke)

Pull request description:

  Closes #17504

  Now that we no longer provide downloads for i686 on our website (https://bitcoincore.org/en/download/), there is no need to build them by default.

  i686 can still be built in depends (tested by ci/travis) and in guix/gitian by setting the appropriate `HOSTS`.

ACKs for top commit:
  practicalswift:
    ACK fae9084ac5b10f94bdee54853d307838c4254e9c -- patch looks correct
  dongcarl:
    ACK fae9084ac5b10f94bdee54853d307838c4254e9c patch looks correct
  laanwj:
    Code review ACK fae9084ac5b10f94bdee54853d307838c4254e9c
  hebasto:
    ACK fae9084ac5b10f94bdee54853d307838c4254e9c, I have reviewed the code and it looks OK, I agree it can be merged.

Tree-SHA512: b000c19a2cd2a596a52028fa298c4022c24cfdfc1bdb3795a90916d0a00a32e4dd22278db93790b6a11724e08ea8451f4f05c77bc40d1664518e11a8c82d6e29
2023-02-20 09:09:23 -06:00
fanquake
359c0fd62b Merge #17933: guix: Pin Guix using guix time-machine
88c83636d5a56bd9551577139786bdd3e74852c2 guix: Update documentation for time-machine (Carl Dong)
e6050884fdabfa6e51e6afce2041d91e60a5adec guix: Pin Guix using `guix time-machine` (Carl Dong)

Pull request description:

  An alternative to #16519, pinning our version of Guix and eliminating a `guix pull` and changing the default Guix profile of builders.

  I think this method might be superior, as it:
  - Eliminates the possibility of future changes to the `guix environment` command line interface breaking our builds
  - Eliminates the need to set up a separate channel repo

  It is a more general pinning solution than #16519.

  -----

  The reason why I didn't originally propose this is because `guix time-machine` is a recent addition to Guix, only available since `f675f8dec73d02e319e607559ed2316c299ae8c7`

ACKs for top commit:
  fanquake:
    ACK 88c83636d5a56bd9551577139786bdd3e74852c2

Tree-SHA512: 85e03b0987ffa86da73e02801e1cd8b7622698d70c4ba4e60561611be1e9717d661c2811a59b3e137b1b8eef2d0ba37c313867d035ebc89c3bd06a23a078064a
2023-02-20 09:09:23 -06:00
MarcoFalke
8a293adb88 partial Merge #17351: doc: Fix some misspellings
ac831339cbfa65b1f7576c53b5d9a94841db9868 doc: Fix some misspellings (randymcmillan)

Pull request description:

  Here is a more thorough lint-spelling update.
  This PR takes care of easy to fix spelling errors to clean up the linting stages.
  There are misspellings coded into the functional tests.
  That is a whole separate job within itself.

ACKs for top commit:
  practicalswift:
    ACK ac831339cbfa65b1f7576c53b5d9a94841db9868 -- diff looks correct

Tree-SHA512: d8fad83fed083715655f148263ddeffc6752c8007d568fcf3dc2c418ccd5db70089ce3ccfd3994fcbd78043171402eb9cca5bdd5125287e22c42ea305aaa6e9d
2023-02-20 09:09:23 -06:00
fanquake
9eb0995a88 Merge #16810: guix: Remove ssp spec file hack
0065ead5ebdad9f743d89ee2558c93253a597967 contrib: guix: Remove ssp spec file hack (Carl Dong)
0093a5869a30797ccb139d0cb5a0427d3f3c2d94 contrib: guix: More robust search paths, add checks (Carl Dong)

Pull request description:

  See commit messages for more details

ACKs for top commit:
  fanquake:
    ACK 0065ead5ebdad9f743d89ee2558c93253a597967

Tree-SHA512: fde04005fb31cd4b75b80da4936a7c394f63f0b3bdcc33c20c99e05604a63efd9c850a8d097030ff0bf4b4e83f1a9997fc4621ce291ebcecd8397893447600a7
2023-02-20 09:09:23 -06:00
Wladimir J. van der Laan
aec7441ac2 Merge #15277: contrib: Enable building in Guix containers
751549b52a9a4cd27389d807ae67f02bbb39cd7f contrib: guix: Additional clarifications re: substitutes (Carl Dong)
cd3e947f50db7cfe05c05b368c25742193729a62 contrib: guix: Various improvements. (Carl Dong)
8dff3e48a9e03299468ed3b342642f01f70da9db contrib: guix: Clarify SOURCE_DATE_EPOCH. (Carl Dong)
3e80ec3ea9691c7c89173de922a113e643fe976b contrib: Add deterministic Guix builds. (Carl Dong)

Pull request description:

  ~~**This post is kept updated as this project progresses. Use this [latest update link](https://github.com/bitcoin/bitcoin/pull/15277#issuecomment-497303718) to see what's new.**~~

  Please read the `README.md`.

  -----

  ### Guix Introduction

  This PR enables building bitcoin in Guix containers. [Guix](https://www.gnu.org/software/guix/manual/en/html_node/Features.html) is a transactional package manager much like Nix, but unlike Nix, it has more of a focus on [bootstrappability](https://www.gnu.org/software/guix/manual/en/html_node/Bootstrapping.html) and [reproducibility](https://www.gnu.org/software/guix/blog/tags/reproducible-builds/) which are attractive for security-sensitive projects like bitcoin.

  ### Guix Build Walkthrough

  Please read the `README.md`.

  [Old instructions no. 4](https://github.com/bitcoin/bitcoin/pull/15277#issuecomment-497303718)

  [Old instructions no. 3](https://github.com/bitcoin/bitcoin/pull/15277#issuecomment-493827011)

  [Old instructions no. 2](https://github.com/bitcoin/bitcoin/pull/15277#issuecomment-471658439)

  <details>
  <summary>Old instructions no. 1</summary>
  In this PR, we define a Guix [manifest](https://www.gnu.org/software/guix/manual/en/html_node/Invoking-guix-package.html#profile_002dmanifest) in `contrib/guix/manifest.scm`, which declares what packages we want in our environment.

  We can then invoke
  ```
  guix environment --manifest=contrib/guix/manifest.scm --container --pure --no-grafts --no-substitutes
  ```
  To have Guix:
  1. Build an environment containing the packages we defined in our `contrib/guix/manifest.scm` manifest from the Guix bootstrap binaries (see [bootstrappability](https://www.gnu.org/software/guix/manual/en/html_node/Bootstrapping.html) for more details).
  2. Start a container with that environment that has no network access, and no access to the host's filesystem except to the `pwd` that it was started in.
  3. Drop you into a shell in that container.

  > Note: if you don't want to wait hours for Guix to build the entire world from scratch, you can eliminate the `--no-substitutes` option to have Guix download from available binary sources. Note that this convenience doesn't necessarily compromise your security, as you can check that a package was built correctly after the fact using `guix build --check <packagename>`

  Therefore, we can perform a build of bitcoin much like in Gitian by invoking the following:

  ```
  make -C depends -j"$(nproc)" download && \
      cat contrib/guix/build.sh | guix environment --manifest=contrib/guix/manifest.scm --container --pure --no-grafts --no-substitutes
  ```

  We don't include `make -C depends -j"$(nproc)" download` inside `contrib/guix/build.sh` because `contrib/guix/build.sh` is run inside the container, which has no network access (which is a good thing).
  </details>

  ### Rationale

  I believe that this represents a substantial improvement for the "supply chain security" of bitcoin because:

  1. We no longer have to rely on Ubuntu for our build environment for our releases ([oh the horror](72bd4ab867/contrib/gitian-descriptors/gitian-linux.yml (L10))), because Guix builds everything about the container, we can perform this on almost any Linux distro/system.
  2. It is now much easier to determine what trusted binaries are in our supply chain, and even make a nice visualization! (see [bootstrappability](https://www.gnu.org/software/guix/manual/en/html_node/Bootstrapping.html)).
  3. There is active effort among Guix folks to minimize the number of trusted binaries even further. OriansJ's [stage0](https://github.com/oriansj/stage0), and janneke's [Mes](https://www.gnu.org/software/mes/) all aim to achieve [reduced binary boostrap](http://joyofsource.com/reduced-binary-seed-bootstrap.html) for Guix. In fact, I believe if OriansJ gets his way, we will end up some day with only a single trusted binary: hex0 (a ~500 byte self-hosting hex assembler).

  ### Steps to Completion

  - [x] Successfully build bitcoin inside the Guix environment
  - [x] Make `check-symbols` pass
  - [x] Do the above but without nasty hacks
  - [x] Solve some of the more innocuous hacks
  - [ ] Make it cross-compile (HELP WANTED HERE)
    - [x] Linux
      - [x] x86_64-linux-gnu
      - [x] i686-linux-gnu
      - [x] aarch64-linux-gnu
      - [x] arm-linux-gnueabihf
      - [x] riscv64-linux-gnu
    - [ ] OS X
      - [ ] x86_64-apple-darwin14
    - [ ] Windows
      - [ ] x86_64-w64-mingw32
  - [ ] Maybe make importer for depends syntax
  - [ ] Document build process for future releases
  - [ ] Extra: Pin the revision of Guix that we build with with Guix [inferiors](https://www.gnu.org/software/guix/manual/en/html_node/Inferiors.html)

  ### Help Wanted

  [Old content no. 3](https://github.com/bitcoin/bitcoin/pull/15277#issuecomment-483318210)

  [Old content no. 2](https://github.com/bitcoin/bitcoin/pull/15277#issuecomment-471658439)

  <details>
  <summary>Old content no. 1</summary>
  As of now, the command described above to perform a build of bitcoin a lot like Gitian works, but fails at the `check-symbols` stage. This is because a few dynamic libraries are linked in that shouldn't be.

  Here's what `ldd src/bitcoind` looks like when built in a Guix container:
  ```
  	linux-vdso.so.1 (0x00007ffcc2d90000)
  	libdl.so.2 => /gnu/store/h90vnqw0nwd0hhm1l5dgxsdrigddfmq4-glibc-2.28/lib/libdl.so.2 (0x00007fb7eda09000)
  	librt.so.1 => /gnu/store/h90vnqw0nwd0hhm1l5dgxsdrigddfmq4-glibc-2.28/lib/librt.so.1 (0x00007fb7ed9ff000)
  	libstdc++.so.6 => /gnu/store/4sqps8dczv3g7rwbdibfz6rf5jlk7w90-gcc-5.5.0-lib/lib/libstdc++.so.6 (0x00007fb7ed87c000)
  	libpthread.so.0 => /gnu/store/h90vnqw0nwd0hhm1l5dgxsdrigddfmq4-glibc-2.28/lib/libpthread.so.0 (0x00007fb7ed85b000)
  	libm.so.6 => /gnu/store/h90vnqw0nwd0hhm1l5dgxsdrigddfmq4-glibc-2.28/lib/libm.so.6 (0x00007fb7ed6da000)
  	libgcc_s.so.1 => /gnu/store/4sqps8dczv3g7rwbdibfz6rf5jlk7w90-gcc-5.5.0-lib/lib/libgcc_s.so.1 (0x00007fb7ed6bf000)
  	libc.so.6 => /gnu/store/h90vnqw0nwd0hhm1l5dgxsdrigddfmq4-glibc-2.28/lib/libc.so.6 (0x00007fb7ed506000)
  	/gnu/store/h90vnqw0nwd0hhm1l5dgxsdrigddfmq4-glibc-2.28/lib/ld-linux-x86-64.so.2 => /usr/lib64/ld-linux-x86-64.so.2 (0x00007fb7ee3a0000)
  ```

  And here's what it looks in one of our releases:
  ```
  	linux-vdso.so.1 (0x00007ffff52cd000)
  	libpthread.so.0 => /usr/lib/libpthread.so.0 (0x00007f87726b4000)
  	librt.so.1 => /usr/lib/librt.so.1 (0x00007f87726aa000)
  	libm.so.6 => /usr/lib/libm.so.6 (0x00007f8772525000)
  	libgcc_s.so.1 => /usr/lib/libgcc_s.so.1 (0x00007f877250b000)
  	libc.so.6 => /usr/lib/libc.so.6 (0x00007f8772347000)
  	/lib64/ld-linux-x86-64.so.2 => /usr/lib64/ld-linux-x86-64.so.2 (0x00007f8773392000)
  ```

  ~~I suspect it is because my script does not apply the gitian-input patches [described in the release process](https://github.com/bitcoin/bitcoin/blob/master/doc/release-process.md#fetch-and-create-inputs-first-time-or-when-dependency-versions-change) but there is no description as to how these patches are applied.~~ It might also be something else entirely.

  Edit: It is something else. It appears that the gitian inputs are only used by [`gitian-win-signer.yml`](d6e700e40f/contrib/gitian-descriptors/gitian-win-signer.yml (L14))
  </details>

  ### How to Help

  1. Install Guix on your distro either [from source](https://www.gnu.org/software/guix/manual/en/html_node/Requirements.html) or perform a [binary installation](https://www.gnu.org/software/guix/manual/en/html_node/Binary-Installation.html#Binary-Installation)
  2. Try out my branch and the command described above!

ACKs for top commit:
  MarcoFalke:
    Thanks for the replies. ACK 751549b52a9a4cd27389d807ae67f02bbb39cd7f
  laanwj:
    ACK 751549b52a9a4cd27389d807ae67f02bbb39cd7f

Tree-SHA512: 50e6ab58c6bda9a67125b6271daf7eff0ca57d0efa8941ed3cd951e5bf78b31552fc5e537b1e1bcf2d3cc918c63adf19d685aa117a0f851024dc67e697890a8d
2023-02-20 09:09:23 -06:00
Kittywhiskers Van Gogh
f42288c984 partial bitcoin#19077: Add sqlite as an alternative wallet database and use it for new descriptor wallets 2023-02-07 10:53:33 -06:00
MarcoFalke
405ab79e29
Merge #18879: valgrind: remove outdated suppressions
d7120f7f78cda5ed1ab91f83e9b546de68dbee47 valgrind : remove duplicate BCLog::Logger suppression (fanquake)
708e3c7e85a666d5b8da8638a819c0f3973fcca4 valgrind: remove rest_blockhash_by_height suppression (fanquake)

Pull request description:

  708e3c7e85: `Suppress rest_blockhash_by_height` should no longer be needed after #18785.

  d7120f7f78 : Removes a duplicate `Suppress BCLog::Logger::StartLogging()` suppression that was added in #17770.

ACKs for top commit:
  MarcoFalke:
    ACK d7120f7f78cda5ed1ab91f83e9b546de68dbee47
  practicalswift:
    ACK d7120f7f78cda5ed1ab91f83e9b546de68dbee47 -- patch looks correct and valgrind Travis job is happy

Tree-SHA512: 45f5b9fa64bf83cada3cd9ad33c245f660376d5b29f51a2531d83133940090df945f5ef26c5847d6ec024ffab9528d55573c5cf9ca5e73795f9abfc971b3d29b
2023-01-23 12:22:30 -06:00
MarcoFalke
a8820d894f Merge #19474: doc: Use precise permission flags where possible
fab558612278909df93bdf88f5727b04f13aef0f doc: Use precise permission flags where possible (MarcoFalke)

Pull request description:

  Instead of mentioning the all-encompassing `-whitelist*` settings, change the docs to mention the exact permission flag that will influence the behaviour.

  This is needed because in the future, the too-broad `-whitelist*` settings (they either include *all* permission flags or apply to *all* peers) might be deprecated to require the permission flags to be enumerated.

  Alternatively, in the future there could be an RPC to set the net permission flags on an existing connection, in which case the `-whitelist*` terminology is of no help.

ACKs for top commit:
  jnewbery:
    reACK fab558612278909df93bdf88f5727b04f13aef0f
  fjahr:
    Code review ACK fab558612278909df93bdf88f5727b04f13aef0f
  jonatack:
    ACK fab558612278909df93bdf88f5727b04f13aef0f

Tree-SHA512: c7dea3e577d90103bb2b0ffab7b7c8640b388932a3a880f69e2b70747fc9213dc1f437085671fd54c902ec2a578458b8a2fae6dbe076642fb88efbf9fa9e679c
2023-01-19 23:37:39 -06:00
Kittywhiskers Van Gogh
07bd5977e5 partial bitcoin#23511: require glibc 2.18+ 2023-01-18 19:02:39 -06:00
Kittywhiskers Van Gogh
d37f30ce46 merge bitcoin#20476: Add test for ELF symbol-check 2023-01-18 19:02:39 -06:00
Kittywhiskers Van Gogh
f62fd6f51f merge bitcoin#22244: Correctly extract symbol versions in symbol-check 2023-01-18 19:02:39 -06:00
Kittywhiskers Van Gogh
29c885c9f7 merge bitcoin#20434: Parse ELF directly for symbol and security checks 2023-01-18 19:02:39 -06:00
Kittywhiskers Van Gogh
ef69317b59 merge bitcoin#19525: add -Wl,-z,separate-code to hardening flags 2023-01-18 19:02:39 -06:00
UdjinM6
828fac4c14 chore: add missing copyrights via copyright_header.py insert 2023-01-13 00:49:04 +03:00
UdjinM6
498e8c5017 chore: run copyright_header.py update 2023-01-13 00:49:04 +03:00
UdjinM6
a86b891d20 chore: update copyright_header.py 2023-01-13 00:49:04 +03:00
Kittywhiskers Van Gogh
816edc4e4f build: add exclusions to linting scripts and filters 2022-12-01 00:51:08 +05:30
Kittywhiskers Van Gogh
305abe91f9
build/depends: commit dashpay/bls-signatures@66ee820f to source tree as vendored (#5077)
* Squashed 'src/dashbls/' content from commit 66ee820fbc

git-subtree-dir: src/dashbls
git-subtree-split: 66ee820fbc9e3b97370db8c164904af48327a124

* build: stop tracking build-system generated relic_conf.h.in

* build: add support for building bls-signatures from local subtree

* build: add exclusions to linting scripts and filters

* build: drop bls-signatures (bls-dash) from depends
2022-11-22 11:34:46 -06:00
PastaPastaPasta
ad88fab80d
test/refactor: upgrade to cppcheck 2.9 and fix warnings (#5049)
* refactor: resolve warnings thrown by cppcheck 2.9

* test: upgrade cppcheck to version 2.9
2022-10-18 13:24:00 +03:00
Wladimir J. van der Laan
8c6fb5622d Merge #17823: scripts: Read suspicious hosts from a file instead of hardcoding
e1c582cbaa4c094d204da34c3b1fdd0d4c557519 contrib: makeseeds: Read suspicious hosts from a file instead of hardcoding (Sanjay K)

Pull request description:

  referring to: https://github.com/bitcoin/bitcoin/issues/17020
  good first issue: reading SUSPICIOUS_HOSTS from a file.
  I haven't changed the base hosts that were included in the original source, just made it readable from a file.

ACKs for top commit:
  practicalswift:
    ACK e1c582cbaa4c094d204da34c3b1fdd0d4c557519 -- diff looks correct

Tree-SHA512: 18684abc1c02cf52d63f6f6ecd98df01a9574a7c470524c37e152296504e2e3ffbabd6f3208214b62031512aeb809a6d37446af82c9f480ff14ce4c42c98e7c2
2022-10-17 15:41:14 -05:00