sikkienl/dash
1
0
Fork 0
mirror of https://github.com/dashpay/dash.git synced 2024-12-24 19:42:46 +01:00
Code Issues Actions Packages Projects Releases Wiki Activity
3d6cc69131
dash/contrib/macdeploy
History
Wladimir J. van der Laan 3d6cc69131 Merge #20333: build: remove native_biplist dependency 
7087440894a9daa7de806c5aa42d83ad60759c65 depends: native_ds_store 1.3.0 (fanquake)

Pull request description:

  `ds_store` [now takes advantage](36fb607940) of Pythons ability to decode binary [plists](https://docs.python.org/3/library/plistlib.html) (since 3.4), so we can drop its biplist dependency.

  The call to `biplist.Data()` in `custom_dsstore.py` doesn't seem to do anything, and from what I can tell can just be removed. i.e:
  ```diff
  diff --git a/contrib/macdeploy/custom_dsstore.py b/contrib/macdeploy/custom_dsstore.py
  index dc1c1882d..e475bc6c3 100755
  --- a/contrib/macdeploy/custom_dsstore.py
  +++ b/contrib/macdeploy/custom_dsstore.py
  @@ -47,6 +47,7 @@ alias.volume.disk_image_alias.target.filename = package_name_ns + '.temp.dmg'
   alias.volume.disk_image_alias.target.carbon_path = 'Macintosh HD:Users:\x00bitcoinuser:\x00Documents:\x00bitcoin:\x00bitcoin:\x00' + package_name_ns + '.temp.dmg'
   alias.volume.disk_image_alias.target.posix_path = 'Users/bitcoinuser/Documents/bitcoin/bitcoin/' + package_name_ns + '.temp.dmg'
   alias.target.carbon_path = package_name_ns + ':.background:\x00background.tiff'
  +assert(biplist.Data(alias.to_bytes()) == alias.to_bytes())
   icvp['backgroundImageAlias'] = biplist.Data(alias.to_bytes())
   ds['.']['icvp'] = icvp
  ```

ACKs for top commit:
  laanwj:
    ACK 7087440894a9daa7de806c5aa42d83ad60759c65

Tree-SHA512: 8ba3cf561937efe4a3daae8b0cb4de3bf9e425b3a9244161b09d94ee2b1bd4c3e21315fa70e495b19a052aabdc1731b3b6f346b63272d72d2762ced83237d02f
2022-03-05 13:48:08 -06:00
..
background.svg Merge #7192: Unify product name to as few places as possible 2017-12-11 08:30:26 +01:00
custom_dsstore.py Merge #20333: build: remove native_biplist dependency 2022-03-05 13:48:08 -06:00
detached-sig-apply.sh partial merge #16327: scripts and tools: Update ShellCheck linter 2021-12-03 18:13:01 +03:00
detached-sig-create.sh partial merge #16327: scripts and tools: Update ShellCheck linter 2021-12-03 18:13:01 +03:00
fancy.plist Merge remote-tracking branch 'bitcoin/0.12' into HEAD 2016-02-06 16:48:04 +03:00
gen-sdk lint: Fix typos flagged by codespell (#4639) 2021-12-29 00:45:54 +03:00
LICENSE Added a copy of the GPLv3 to macdeployqtplus 2011-11-02 19:26:32 +01:00
macdeployqtplus Merge #15257: Scripts and tools: Bump flake8 to 3.7.8 2021-10-05 20:34:01 +03:00
README.md merge bitcoin#20470: Replace genisoimage with xorriso 2021-10-05 08:04:26 +05:30

README.md

MacOS Deployment

The macdeployqtplus script should not be run manually. Instead, after building as usual:

make deploy

During the deployment process, the disk image window will pop up briefly when the fancy settings are applied. This is normal, please do not interfere, the process will unmount the DMG and cleanup before finishing.

When complete, it will have produced Dash-Qt.dmg.

SDK Extraction

Step 1: Obtaining Xcode.app

Our current macOS SDK (Xcode-12.1-12A7403-extracted-SDK-with-libcxx-headers.tar.gz) can be extracted from Xcode_12.1.xip. An Apple ID is needed to download this.

After Xcode version 7.x, Apple started shipping the Xcode.app in a .xip archive. This makes the SDK less-trivial to extract on non-macOS machines. One approach (tested on Debian Buster) is outlined below:

# Install/clone tools needed for extracting Xcode.app
apt install cpio
git clone https://github.com/bitcoin-core/apple-sdk-tools.git

# Unpack Xcode_12.1.xip and place the resulting Xcode.app in your current
# working directory
python3 apple-sdk-tools/extract_xcode.py -f Xcode_12.1.xip | cpio -d -i

On macOS the process is more straightforward:

xip -x Xcode_12.1.xip

Step 2: Generating Xcode-12.1-12A7403-extracted-SDK-with-libcxx-headers.tar.gz from Xcode.app

To generate Xcode-12.1-12A7403-extracted-SDK-with-libcxx-headers.tar.gz, run the script gen-sdk with the path to Xcode.app (extracted in the previous stage) as the first argument.

# Generate a Xcode-12.1-12A7403-extracted-SDK-with-libcxx-headers.tar.gz from
# the supplied Xcode.app
./contrib/macdeploy/gen-sdk '/path/to/Xcode.app'

Deterministic macOS DMG Notes

Working macOS DMGs are created in Linux by combining a recent clang, the Apple binutils (ld, ar, etc) and DMG authoring tools.

Apple uses clang extensively for development and has upstreamed the necessary functionality so that a vanilla clang can take advantage. It supports the use of -F, -target, -mmacosx-version-min, and -isysroot, which are all necessary when building for macOS.

Apple's version of binutils (called cctools) contains lots of functionality missing in the FSF's binutils. In addition to extra linker options for frameworks and sysroots, several other tools are needed as well such as install_name_tool, lipo, and nmedit. These do not build under Linux, so they have been patched to do so. The work here was used as a starting point: mingwandroid/toolchain4.

In order to build a working toolchain, the following source packages are needed from Apple: cctools, dyld, and ld64.

These tools inject timestamps by default, which produce non-deterministic binaries. The ZERO_AR_DATE environment variable is used to disable that.

This version of cctools has been patched to use the current version of clang's headers and its libLTO.so rather than those from llvmgcc, as it was originally done in toolchain4.

To complicate things further, all builds must target an Apple SDK. These SDKs are free to download, but not redistributable. To obtain it, register for an Apple Developer Account, then download Xcode_11.3.1.

This file is many gigabytes in size, but most (but not all) of what we need is contained only in a single directory:

Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk

See the SDK Extraction notes above for how to obtain it.

The Gitian descriptors build 2 sets of files: Linux tools, then Apple binaries which are created using these tools. The build process has been designed to avoid including the SDK's files in Gitian's outputs. All interim tarballs are fully deterministic and may be freely redistributed.

xorrisofs is used to create the DMG.

xorrisofs cannot compress DMGs, so afterwards, the DMG tool from the libdmg-hfsplus project is used to compress it. There are several bugs in this tool and its maintainer has seemingly abandoned the project.

The DMG tool has the ability to create DMGs from scratch as well, but this functionality is broken. Only the compression feature is currently used. Ideally, the creation could be fixed and xorrisofs would no longer be necessary.

Background images and other features can be added to DMG files by inserting a .DS_Store before creation. This is generated by the script contrib/macdeploy/custom_dsstore.py.

As of OS X 10.9 Mavericks, using an Apple-blessed key to sign binaries is a requirement in order to satisfy the new Gatekeeper requirements. Because this private key cannot be shared, we'll have to be a bit creative in order for the build process to remain somewhat deterministic. Here's how it works:

  • Builders use Gitian to create an unsigned release. This outputs an unsigned DMG which users may choose to bless and run. It also outputs an unsigned app structure in the form of a tarball, which also contains all of the tools that have been previously (deterministically) built in order to create a final DMG.
  • The Apple keyholder uses this unsigned app to create a detached signature, using the script that is also included there. Detached signatures are available
  • Builders feed the unsigned app + detached signature back into Gitian. It uses the pre-built tools to recombine the pieces into a deterministic DMG.