Imaging is Dead… Long Live the Installer!

The writing has been on the wall for a long time. With the release of macOS High Sierra, Apple has finally confirmed that imaging is dead.

Apple doesn’t recommend or support monolithic system imaging for macOS upgrades.

I have written a book which expands on this topic and is regularly updated. Please check it out: “macOS Installation for Apple Administrators“

Update 2019-07-16: While most of the information in this post is still relevant,  I wrote a new, updated post regarding the macOS 10.15 Catalina Upgrade here: “Imaging is still dead“

The Final Nail

The final nail in the coffin for imaging is this support article: Upgrade macOS on a Mac at your institution

It states the limitations to installing and upgrading macOS with High Sierra:

• the Mac being installed or updated must be connected to the internet
• installations and updates cannot be done on external devices, like those connected via Target Disk Mode, Thunderbolt, USB, or Firewire
• there are four supported methods of installing macOS High Sierra
  • the macOS Installer application
  • a bootable installer on an external drive, created with createinstallmedia
  • install or upgrade from the macOS Recovery System
  • a NetInstall image built with Apple’s System Image Utility

These methods are also re-iterated in this section in the macOS Deployment Reference

Some of the features have changed since this article was written. You can find an updated post for macOS 10.13.4 here.

Why?

Apple’s stated reason for requiring the installer is to ensure that a Mac’s firmware is all up to date and matches the OS installed on it.

Only the macOS Installer can download and install the firmware update. Firmware updates can’t be done on external devices, like those connected via Target Disk Mode, Thunderbolt, USB, or Firewire.

This is especially important in High Sierra, because to boot into a system on an APFS formatted (or converted disk) the Mac’s firmware needs to be able to mount and read APFS. The firmware that was installed with 10.12 or earlier is not able to read APFS volumes and. When you image a Mac with High Sierra and APFS without updating the firmware, you will get the question mark at boot, because the firmware cannot find a system.

However, the EFI, which manages (among other things) the boot process is not the only “firmware” that needs to be managed on your Mac. Many of the hardware components in your Mac, such as the SSD, the power controller (SMC) and the TouchBar controller on the new MacBooks Pro, have their own firmware that needs to be installed and updated.

Apple has been increasing the protection of the vital parts of the system and hardware. The firmware in these components cannot just be changed by any process. Only the Apple macOS Installer application has sufficient privileges and entitlements to perform these updates. The installer process has to run on the Mac itself, it cannot run over target disk mode.

Future Mac hardware might introduce even more components that require firmware.

On iOS, a secure boot chain prevents tampering with the system on a device after it has been installed. The secure boot chain also prevents replacing the system with an image from another device.

It is conceivable that Apple wants to implement a secure boot system on future Macs as well. Current Macs probably do not have the hardware required to implement this. (The TouchBar MacBooks Pro have a Secure Enclave chip like the iPhone and iPad and might already have the necessary pieces in place.)

So now what?

This has been coming for a long time. Even though APFS is not, as originally predicted, the direct culprit. It is still end-of-the-line for imaging.

However, the news is not entirely dire. Apple has been surprisingly forthright about the direction they want to go. While the documentation is a bit lacking, there are instructions on what the solutions for Mac System Adminstrators should be.

The four supported means of installing and upgrading macOS and the firmware for Macs are a clear direction of what needs to be done. However, Apple is a bit shy on how administrators can and should implement them.

There are a few options:

Put the Burden on the Users

This will work in some deployments where users are in control of their Macs. Either a full BYOD (Bring Your Own Device) scenario, or one where devices provided by the organisation are in full control of the user.

Even when the devices are enrolled in an MDM, users are still administrators and in control. Administrators can use reporting tools, to determine which Macs are capable of installing High Sierra and not upgraded yet.

You can even use reporting tools to gather information on the firmware and whether it matches the latest version.

You can then instruct users with email or notifications to download the High Sierra installer and initiate the upgrade themselves. You should warn them to have a current backup and that the process might take some time, so it should be run overnight.

If you have a software management system in place, you can use that to load the macOS Installer application on the clients and notify the user when it is ready.

The Mac App Store only downloads a “stub” installer application which is then filled in with an extra download. If you have blocked access to Apple Software Update Servers or redirected clients to a local, managed Software Update Server, clients might not get the complete installer application. Greg Neagle has a great post on this.

To save download time for the users, you can also provide USB/Thunderbolt drives with a bootable installer drive. This might speed things up a bit, though it does not really change the process.

How do we Automate this?

As system administrators we want to automate the process, so that it ideally does not require any human interaction. That way we can replicate the process hundreds and thousands of times.

Ideally, we also want to inject some custom steps into the process. Apple provides two means of achieving both of these steps, and some open source tools are providing solutions as well.

NetInstall

A custom NetInstall set built with Apple’s System Image Utility is one of the supported means of installing and updating macOS (and the firmware) to High Sierra.

You can find System Image Utility in /System/Library/CoreService/Applications. You can customize the process and even add your own installer packages, scripts or profiles. Packages used in System Image Utility have to be Distribution Packages.

Since a NetInstall system is like a Recovery system, you can use scripts to control behavior of your Mac like allowed NetBoot IPs, or ‘User-Approved Kernel Extension Loading’ (UAKEL) here as well.

You can even automate the NetInstall process to a point where, once you have chosen the NetInstall volume (when holding the the option key at boot) the remaining process is without interaction. (Though this is a dangerous choice, as it might simply wipe and re-install Macs. Use the other limitation options such as by MAC address or hardware type to keep this safe.)

NetInstall requires Mac running macOS Server. However, BSDPy can replace a NetBoot/NetInstall server and runs on many platforms, including VMs.

Note: as of 10.13.0 there still seem to be a few bugs with NetInstall on HighSierra. It works mostly but seems exceedingly slow. Also some admins have reported problems with adding mulitple packages, profiles or scripts for configuration. 10.13.1 is already in beta and seeding and you should be testing that.

NetInstall on USB

If you do not have the infrastructure to run a NetInstall or BSDPy server, you can also restore the NetInstall.dmg image that System Image Utility creates to an external drive. When mounted on a Mac, it will the High Sierra installer applications, and double-clicking it will start the proper installation process.

Any additional packages, profiles or scripts will be included with this custom external install application as well.

The startosinstall Command

When you already have a management system (Munki, Jamf, Filewave, etc.) you want to initiate the update process with rules or policies. At first glance the supported means of installing macOS seem to be at odds with managed client workflows, since they require user interaction.

However, there is a tool hidden inside the macOS Installer application (since macOS 10.12 Sierra) called startosinstall. The full path to the tool is

/Applications/Install macOS High Sierra.app/Contents/Resources/startosinstall

Note: I believe it was Rich Trouton who first documented this tool in his notes for WWDC 2016. Since then many admins and open source projects have worked to figure out how to use this tool in the best way.

When you run it with the --usage argument you get the following:

$ /Applications/Install\ macOS\ High\ Sierra.app/Contents/Resources/startosinstall --usage
Usage: startosinstall

Arguments
--applicationpath, a path to copy of the OS installer application to start the install with.
--license, prints the user license agreement only.
--agreetolicense, agree to license the license you printed with --license.
--rebootdelay, how long to delay the reboot at the end of preparing. This delay is in seconds and has a maximum of 300 (5 minutes).
--pidtosignal, Specify a PID to which to send SIGUSR1 upon completion of the prepare phase. To bypass "rebootdelay" send SIGUSR1 back to startosinstall.
--converttoapfs, specify either YES or NO on if you wish to convert to APFS.
--installpackage, the path of a package to install after the OS installation is complete; this option can be specified multiple times.
--usage, prints this message.

Example: startosinstall --converttoapfs YES 

There is also an undocumented --nointeraction flag which can be used to run the tool without any user interaction. This is obviously useful for management systems.

Once you have used to your management system to make sure the macOS Installer application is on the client system, you can execute a script with the startosinstall command to initiate the installation process. Remember that the installation process can take a long time, so it should be initiated by the user in a Self Management portal or run during off-hours for kiosk like Macs in labs or classrooms.

startosinstall --applicationpath /Applications/Install\ macOS\ High\ Sierra.app \
    --agreetolicense \
    --nointeraction

The --converttoapfs [YES|NO] argument allows you to suppress automatic APFS conversion on SSD Macs.

There is also a --volume argument to target the non-boot volume. However, this will only work when SIP is disabled or when you run startosinstall from a Recovery/NetInstall disk.

The --installpackage option allows you to add one or more custom packages that will be installed after the OS installation is complete. This is very useful for customization and cleanup. Packages used with startosinstall --installpackage also have to be Distribution Packages.

Note: even though the usage states that you can repeat the --installpackage argument, as of 10.13.0 only the first package given will run the installation with fail with more than one package. Make that one package count. (Note: edited this paragraph. Thanks to Greg for clarifying.)

Tool Support

• Munki 3 already supports the startosinstall command
• Imagr will in an upcoming release
• other management systems can use the startosinstall command in scripts, after ensuring the macOS Installer application is downloaded.

Is Imaging completely dead?

The imaging tools (like Disk Utility, hdiutil and asr) will work with APFS volumes. However, the support article states:

You can use system images to re-install the existing operating system on a Mac.

So when you need a workflow that requires quick re-imaging, you can use one of the supported methods to install or update the Mac (Firmware and OS) and then use monolithic (or thin) imaging over network or thunderbolt for fast restores. This is useful for scenarios where fast imaging turnaround is required, such as classrooms and labs or loaner laptop setups. However, you have to use extra care to make sure the image system version matches the version that was installed.

Going forward, I expect imaging to be less and less feasible as future Mac hardware and security features will make it harder and harder to use. The times where we could have just one image which will run on all supported Macs might be over as hardware (and the software required to run the hardware) becomes more and more fractured. Note that there is not a unified single ‘iOS’ image/installer for all iOS devices.

Summary

macOS High Sierra 10.13.0 works well for individual users. However, there still are quite a few issues that are relevant for managed deployments. There are many problems with Active Directory and Filevault, NetInstall is slow and adding multiple packages to an installation is broken. Just to mention a few.

Even though it makes sense for some deployments to hold back from High Sierra right now, you will want or have to upgrade soon.

Apple said at WWDC the iMac Pro will ship in December 2017. Its tech specs page, states it will run High Sierra. You can expect it to require High Sierra.

Also, critical security patches might only be pushed for High Sierra.

Imaging is dead. In an unusual move Apple has come right out and said it loud and clear. If you have not done so already, start testing and implementing one of the above strategies right now, so you are ready to move to High Sierra.

Read about more changes with the macOS 10.13.4 updates here.

I have written a book which expands on this topic and is regularly updated. Please check it out: “macOS Installation for Apple Administrators“