Get an Icon for your Mac

A few weeks ago I had a post about getting the “Marketing Name” for a Mac.

At that time I was also trying to get an icon or image file for the current Mac model, but could not find a way to do it.

Since then I have found that the AppKit framework provides a method to get an image for the Mac.

[NSImage imageNamed: NSImageNameComputer] # Objective-C

NSImage(named: .computer) # Swift

To get this image data into a file requires some passing through other classes. However, this is possible in Python on macOS. (I had some trouble, but figured it out with some help in the MacAdmins Slack #python channel, thanks!)These are the posts that were recommended reading or watching:

In case you need an image file for the Mac, here is the code. It will generate a 512px image for the current Mac. The two lines you may want to change are line 7 for the size of the image and line 16 for the filename.

Update: improved version here (not by me)

NetInstall is Dead, too

I have written a book which expands on this topic and is regularly updated. Please check it out: “macOS Installation for Apple Administrators

Tim Perfitt of Twocanoes Software (Winclone, SD Clone, etc.) got an iMac Pro.

For obvious reasons he immediately looked at the details of the new boot process, and has found some details that were speculated or unknown so far. Most of this happened on Twitter, which is a quite hard to put together afterwards, so here is a summary: (there were several members of the Mac Admin community involved, thanks to all of them!)

Update: Tim Perfitt now as an excellent detailed post on his findings here.

There are probably a few more details which will come out as other admins get their iMac Pros in the following days and weeks. But this gives us enough confirmation of facts to know:

NetBoot is dead!

Don’t Panic!

So the news of NetBoot’s demise has not been exaggerated. Also it is to be expected that all new hardware from Apple going forward will have Secure Boot and probably not NetBoot.

MacAdmins will all need to plan ahead and look at the options that are on the table for Mac management going forward.

There is speculation that the current TouchBar MacBook Pros might get Secure Boot added in a future update to 10.13. Even if that is not the case, then it is a safe assumption that future Mac releases will contain the T2 system controller or something similar and have the same Secure Boot features (or lack thereof).

Deployment Strategies Going Forward

Device Enrollment Program and Mobile Device Management (DEP + MDM) is certainly Apple’s deployment method of choice. They have been pushing to this for a few years now and it has also been the way to manage iOS devices.

DEP is a process where a new device (iOS or Mac) is registered to your organization at purchase and you can assign it to your Mobile Device Management server through Apple’s website.

At it’s very first boot, the Mac will check with Apple’s DEP servers and get the MDM’s information, register with the MDM and then the management settings take over, adding configuration, software and, with some management systems, local tools to install and manage non-AppStore software.

When a Mac’s system volume is erased and macOS is re-installed the process starts over, keeping the Mac managed by the same MDM.

Apple has also made DEP+MDM a requirement to manage Kernel Extensions without user interaction. Furthermore, Apple states that going forward, the “approved” level of MDM (either by DEP or explicit user interaction) will be used for more configurations in the future.

This is similar to “supervised” iOS devices. However, Apple provides two means to supervise an iOS device: with DEP and by manually connecting an iOS device to a Mac with Apple Configurator. The process with Configurator can be automated, aside from the manual connection. On macOS the manual (“user-approved” MDM enrollment) cannot be automated and cannot even be performed over remote control.

In general, DEP+MDM works well. It enables certain management styles and workflows that were not possible before. An organization can order a device from Apple or a reseller and have it sent directly to an employee. When the employee unboxes the new device it is registered with the organization’s MDM and receives configuration profiles and software, even when the device is off-site.

Apple and MDM vendors like to call this workflow “zero-touch” deployment, because the IT department does not have to touch the device. This is a great improvement for many Mac administrators.

However, there are a few downsides to DEP+MDM:

External Dependency

Apple’s DEP servers are an external dependency and a single-point-of-failure in the deployment workflow. There were a few outages of the DEP system this year. Even worse, Apple does not include the DEP service in their status overview page. This leaves Mac admins wondering if a problem is on their side or with Apple.

DEP availability

DEP is not available in all regions where Macs are sold. With imaging and NetInstall off the table, this is leaves only manual installation and MDM enrollment/approval for management.

Also, a client has to be online and the network to have access to Apple’s servers. This requires un-proxied access to Apple’s 17.* IP range. However, especially when you are outside of the US, the processess at installation may attempt to connect to other IP addresses as well.

With Apple Configurator an administrator can also add existing iOS 11 devices into DEP, not just new ones. This option is not available for existing Macs.

User Interaction

DEP + MDM allows to automatically enroll devices without requiring IT to touch a device. However, the process is not automated. There has to be a user present to interact at several points with the Mac for the initial setup. While profiles can manage and reduce the user input required during setup, there are a few steps you cannot automate away.

Also the enrollment process will only install the management tools then show the user the desktop. Actual installation of software packages takes place in the background and might take a long time. This can leave the user confused as to what is going on. Certain management options, such as enabling FileVault may require a logout or restart, interrupting the user with whatever they started to do on their new Mac or leaving the Mac in an unsecure state until the user restarts.

This downside of DEP is so glaring that many open source solutions have sprung up to provide a user interface for the post-DEP initial configuration cycle.

(I am probably missing some, let me know!)

This innovation and initiative of individual admins and the community as a whole is admirable. Thanks to all who provide!

Either way, administrators using DEP + MDM have to be aware of the time required for the download and installation of large software packages and choose which pieces are absolutely required and which can be deferred to be installed later at a time of the user’s choice through a self service portal.

Software and Configuration Management

DEP handles the initial connection to the MDM. The MDM can push and enforce profiles to control some settings. The MDM can also initiate installation of (Mac) App Store software through the Volume Purchase Programm (VPP).

To manage software and configurations that are not in the Mac App Store or not supported by configuration profiles, administrators need to install a local tool on the client system.

The MDM protocol provides a tool called InstallApplication that will instruct a client Mac to download a pkg file and install it. For example, the Jamf Pro management suite uses this to install the jamf binary tool, which then can take over and perform many other management tasks, which the MDM system does not provide.

Some management systems (so far I know of SimpleMDM and AirWatch, let me know if I missed any) allow admins to provide their custom installer to install a local management tool (e.g. Munki, Puppet)

Notably, Apple’s reference MDM implementation, Profile Manager (part of the macOS Server app) does not allow for custom installs.

Erik Gomez has done outstanding work documenting his experiences with this process. The entire series is worth reading. but if you want to catch up quickly, the recent posts have a good summary of the status quo and a real-world implementation.

Offboarding, Re-installation and Re-purposing

Once the initial configuration is complete, MDM+VPP+management system will take care of installations and software updates. However, there are situations, where you will want to ‘nuke and pave’ or ‘wipe and re-install.’

There are many reasons an admin may want to do this, most of them involve ‘configuration drift.’ I.e. over time as more and more software gets installed and configured on a given system, errors and conflicts pile up and cause problems. At this point it is usually easier to ‘nuke and pave’ or ‘erase and install’ than to track down the actual conflicts.

In an ideal world, all the configuration owned by a user would be exclusively in that user’s home directory and you would only have to delete and recreate that user, rather than the entire system. However, we do not live in that world.

Many pieces of software store configuration in central locations, but still assume that these central locations are writable by the user. In most setups this is the case, because by default users are admins on macOS. However, this spreads configuration and other data all through the system, making it impossible to isolate all changes.

With imaging and NetInstall, admins could use the same workflow for the initial installation and configuration than for subsequent-installations.

Furthermore, the process could be automated to the point were no local interaction was necessary, or just the minimal interaction of someone restarting a Mac and holding the ‘N’ key. From then on all steps could run fully automatically and without interaction. Further more, imaging with block copy was fast.

With High Sierra, Imaging is not supported anymore, except in very specific circumstances. Automated Installations with NetInstall are broken. And with the iMac Pro the option for NetInstall goes away completely. (Which may explain why automated NetInstall was not fixed in High Sierra.)

This leaves manually booting to (Internet) Recovery, erasing the startup volume in Disk Utility and re-installing macOS as the only means of re-installing a Mac. After the installation DEP should re-connect the Mac with the MDM and management should take over.

However, you cannot completely automate the DEP interaction, so after waiting for ~30minutes for the installation to complete, someone has to confirm a few dialogs before managed installation can kick in and do the rest of the work. All this interaction is time-consuming and error-prone.

“Erase all Contents”

On iOS, you rarely have to re-install the system. Instead there is a function ‘Erase all Contents and Settings’ which restores a device to a clean unconfigured state, from which DEP and then the MDM can take over. You can even send the wipe command over the air with an MDM. (On iOS you also have to manually confirm a few dialogs before DEP and MDM can take over, but the entire process is much faster.)

Until Apple provides this feature on macOS locally and remotely, admins who rely on fast restores either have to stay on Sierra, postpone new hardware purchases, or revisit and redesign their workflows.

This mostly affects education customers with lab deployments. Other large Mac deployments were a Mac is “owned” by a single user, are less affected by this.

Sidenote on Mac App Store and VPP

Mac App Store applications have to be sandboxed, which means they can’t even access all of the user home directory, only their designated sandbox. Managing these applications and the data is much more manageable than other Mac applications and tools where anything goes.

On iOS, the App Store and VPP are the only means of distributing applications, and this is much simpler and manageable. However, the App Store rules prohibit entire classes of tools and services. While the Mac App Store enforces similar rules, users can still download and install applications and tools outside of the Mac App Store. For most Mac users, this is the defining advantage of macOS.

However, this higher complexity of software and deployment methods, requires more complex deployment and configuration workflows.

MDM + VPP cannot handle this complexity, which is why management systems, such as Jamf Pro, Filewave, and Munki, exist and need to exist for macOS management.

Also I need to say that not all software and installers need to be as complex as they are. Most software that comes with complex installation tools are unnecessarily complex and error prone. Often the developers are just taking a cheap shortcut, by assuming the current user has write access to the application bundle, or has admin privileges, etc.

However, there are still entire classes of professional software, that even if they did simplify their applications and installers, would not currently be allowed in the Mac App Store (IDEs and developer tools, hardware drivers, certain virtualization software features, anything that needs root access, etc.)

Also the App Stores (on macOS and iOS) have features, that cannot be purchased or distributed by VPP. In-App-Purchases and subscriptions cannot be purchased or distributed. Recently, Pre-orders for a special price were added as a feature to the App Stores, but these can also not be used for VPP.

Overall, I would love if all software were available in the (Mac) App Store and could be managed and distributed with VPP, but we are a long way from that reality.

Why all this?

By now it seems fairly obvious that Apple wants to get macOS system security to a point where only Apple can ever affect and change system software and firmware. That is a worthwhile goal. It means that your data is secure on an encrypted drive, they decryption key is locked in the secure enclave, but Apple can design solutions like TouchID and FaceID to unlock everything quickly.

To close the loop for all this security, the system needs to be able to verify and confirm that the software running the system (both the OS and firmware) are up to date and in their original state.

Imaging, NetBoot and NetInstall bypass most of this security. I believe it could be possible to create a networked installation workflow with all the security in mind, but it might just not be worth the effort. Apple seems to think this is not worth doing right now.

And remember that imaging and NetInstall are not valuable in themselves, but they are valuable as tools to achieve something useful, namely: automated installation and configuration of Macs.

DEP + MDM + VPP gets us there in many situations. In many use cases, DEP allows for workflows that were not possible before. Other technologies in macOS High Sierra (snapshots) promise some more useful tools, but they are not quite there yet.

Right now the gap between what we currently use as admins and what will come down the road is getting really wide.

Where to go from here?

There are two things a system administrator needs to balance:

  • provide a stable and efficient environment to manage the computers, software, configurations, and users
  • adapt the environment and workflows for new and future requirements and technologies

These to goals are often at odds and balancing them is a circus act in the best of times. Right now Apple is making our collective lives harder by shaking the rope we are standing on and throwing a few new balls in to the juggling act at the same time.

Apple and the MDM vendors are providing a powerful new solution DEP + MDM which works well for some deployement styles (1-to–1 deployments).

History has shown, that going against Apple’s vision of how their devices should be used, will not result in a smooth experience. Take a good look at your deployments using imaging and NetInstall: might a different deployment scenario work?

In education, labs are often used because certain software is too complex or expensive to be provided to all laptops. In this case virtualization, switching to another software solution or different OS might be a solution. (And please let your Apple rep know you are considering switching to another OS. That is the great leverage we have on Apple to support better workflows.)

To buy some time, you could hold on to Sierra for a while longer. Right now, all Macs except the iMac Pro still support Sierra. As new hardware gets released next year, your options will dwindle. Maybe your organization can accelerate or postpone purchases with that in mind. This cannot last forever, but buy you some time.

However, any new Mac releases will, like the iMac Pro, require High Sierra and (most probably) have the same Secure Boot features. How will you support those when they are purchased? The high price of the iMac Pro might discourage purchases, but for how much longer. You will need to have an answer in place.

(Also, this is great argument that you need an iMac Pro for testing, now… 🙂 )

Your answer may very well be, that you will have to accept the extra manual affort required to (re-)install High Sierra based Macs. However, then you had better have an idea of how much more effort and time will be required, to justify the extra workload to your organization.

Test, test, test!

Do you have a DEP + MDM solution in place? Did you get the budget for it? Are you testing deployment workflows with it? For the past year and more, the writing has been on the wall that this is the way to go. If you haven’t started on this by now, you really, really have to.

Do you have an idea/solution on how to smoothen the new deployment workflow for you or your users? Whether it is just an idea or a finished workflow, please discuss and share it in the MacAdmin community. The MacAdmins Slack is a great place to start. Maybe someone in the community will figure out how to use APFS snapshots to quickly and reliably restore a Mac to a well-known state before Apple does.

There are already someinterestingideas out there.

Maybe you’ve already done all this and found a setup that works for you. Well done! This would be a great time to present your solution and how you got there at a Mac Admin meeting or conference. Many other admins would love to learn from you. (Or just write a blog post.)

Talk with your Apple Reps, file bugs, etc. Don’t expect Apple to bring back imaging or NetInstall, but do point out the shortcomings of Apple’s solutions going forward.

The orchestration for Apple to get the new hardware and software components and pieces in place must be enormous. Some pieces take longer and with patience we will see how everything fits together.

We are living in interesting times!

Happy New Year 2018!

I have written a book which expands on this topic and is regularly updated. Please check it out: “macOS Installation for Apple Administrators

iMac Pro Implications for Mac Admins

The first iMacs Pro will ship this week to some lucky buyers, just in time to keep Apple’s promise of shipping this year.

Now that we have all gotten over the sticker shock when you max out the configuration in the Store, what does the new tech in iMac Pro mean for admins?

Secure Boot

First is the new secure boot. iMac Pro comes with Secure Boot enabled and External Boot disabled. You can disable (or moderate) the settings in the new ‘Startup Security Utility’ in Recovery.

With secure boot enabled, a Mac will verify the integrity of the OS and confirm with Apple before booting. It may require an update to be installed before continuing to boot.

Somewhat surprisingly, Secure Boot on the iMac Pro will verify the integrity of a BootCamp/Windows installation as well as macOS. (The continued persistence of BootCamp makes me wonder what Apple uses it for internally.)

The support article seems to imply that on the strongest setting, the iMac Pro might force an update before you can boot. We will have to wait and see how far back Apple will “trust” older versions of macOS.

By default an iMac Pro will not boot from an external device. This setting can be changed in the ‘External Boot’ area of the Startup Security Utility.

You can still boot to the Startup Manager with the option key but when you select an external drive you will get an error message. You can only select internal drives with the option key.

Both of theses settings can probably only be disabled manually in Recovery mode. This renders most automated installation and imaging procedures useless. Also the support article states that you have to enter a local administrator password to change the setting. This can also be difficult in settings where a tech or admin might not know a local password.

NetBoot

Prohibiting External Boot will (probably) also prohibit NetBoot and NetInstall. However, Apple updated their support article “Create a NetBoot, NetInstall, or NetRestore image” with the note:

iMac Pro computers don’t support starting up from network volumes.

Also the support article “Mac startup key combinations” has added this to the description of the ‘N’ key:

iMac Pro doesn’t support this startup key.

It is as of yet unclear if this means that iMac Pro will not NetBoot under any circumstances or if it will NetBoot, but not in the default configuration and you have disable the boot security first.

The phrasing in the articles seems clear, but it may be an error/omission. If you happen to get your hands on an iMac Pro and can test, NetBoot/NetInstall, please let me (and everybody else) know.

The other question that remains is whether Internet Recovery still works on the iMac Pro. There has not (yet) been an amendment to the Internet Recovery support article. Internet Recovery is a form of NetInstall as well, albeit with a different discovery method.

Imaging is dead and NetInstall is not doing so well

So, as predicted, the iMac Pro puts yet another nail in the coffin of imaging. You will have to run the iMac Pro in a lowered security mode, for it to accept an OS that was not installed on itself and verified by the internal T2 system controller chip.

While it is still possible to disable the boot security, this has to be done manaully. There is no way to automate the deactivation, much like you cannot automate disabling SIP.

Finally, NetInstall might not work at all, even when the boot security is disabled. And even if NetInstall does still work on the iMac Pro, NetInstall is still quite broken in High Sierra: additional pkgs have to be in just the right format to work, automated installations are broken, and you cannot initiate a NetInstall remotely through a script or the management system, but have to be physically at the machine and hold the ‘N’ key. (all of these affect all Macs, not just the iMac Pro)

And even when you have managed to get all of these to work, then new security like UAKEL and UAMDM might still require an administrator to touch all the machines again after re-imaging.

“Zero-touch” deployment

When you consider the standard use case, where a Mac is in possession of a single user (whether it is owned by that user or organisation) then most of these problems are fairly easy to work around with some user guidance and education. DEP enforces enrollment and tools like SplashBuddy and DEPNotify can make the process more understandable for the user.

“Zero-touch” deployment in this case means that the IT department will not have to touch the device. Even though you can automate much of the configuration, the enrollment is not entirely automatic, the process still requires the user to be at the Mac and fill in or confirm some dialogs.

However, for other deployment scenarios, especially general access labs in education, this breaks exisiting workflows. You never know what the users (and applications) are going to do to a system, even if they don’t have adminstrative privileges. Re-imaging rather than figuring out which configuration is broken is a quick and efficient remediation for many problems.

Many professional software packages are notoriously hard to install in an automated fashion and even harder to de-install cleanly. In addition, this kind of software tends to have very strict licensing terms and high prices. “Wipe and re-install” is a simple and fast workflow to ensure software and drivers are removed cleanly and repurpose a Mac (or an entire Lab of Macs) for a different task. (e.g., switch a video or audio lab to an lab with engineering and math software) Many admins have fully automated touch free workflows that can be started remotely through ssh, Apple Remote Desktop or a management system.

Not only do all of these workflows have to be re-visited and re-built without imaging, but they will not be able to run without user interaction. Without NetInstall (or if NetInstall remains broken) the user interaction may be non-trivial.

To wipe and re-install an iMac Pro, an admin has to boot to Recovery, manually erase the drive in Disk Utility and then start the installation process. The tech or admin will have to know and enter a local administrator password. Even with DEP, there are a few dialogs after the installation that need to be confirmed manually before DEP and any automation from the management system can start their work.

True “Zero-touch”, where no-one has to physically touch the Mac, (re-)deployment is not possible with Apple’s currently supported toolset for High Sierra and iMac Pro.

The Missing Piece

If macOS had an “Erase All Content and Setting” option like iOS does, then you could do a quick reset and with DEP + management system quickly restore a Mac to the previous (or a new) configuration. On iOS this is achieved by keeping the system on a separate volume from apps and user data. This separation would not be quite so easy on macOS, but with APFS snapshots the system could create (and preserve) a snapshot after a clean installation and provide hooks for scripts and management systems to restore to that.

It is quite frustrating that this option does not yet exist. Apple is removing older workflows from the toolset without providing a functioning alternative. If Apple decides to implent this function in macOS and enable its automation from an MDM, then you have the best of both worlds, the advanced security and automation and management for admins!

Make Noise

Apple seems to be unware of or indifferent to these methods and workflows. Most enterprise customers might not be affected by them. Those customers that are, need to let Apple know through the usual means: your sales reps, your support contact (if you have one) and by filing bugs.

If you are at an instituition that is considering to buy a classroom full of iMacs Pro you will have a large financial leverage with this deal. So let your sales reps and engineers know of your issues, but also be understanding that they might not have a solution for you right away.

Even so, DEP and MDM will be a major part of whatever solution you will have to use in the future. If you have not started working on your implementation yet, there is no time like the present.

Maybe you can use this article to convince your management to purchase an iMac Pro so you can test it. If that actually works, let me know. 😉

Interesting Software on Sale in the AppStores

I am working on a post on the iMac Pro, but then Apple dropped a few interesting support articles and I have to re-write the entire thing.

Until then, I found a few interesting sales going on the App Stores (iOS and Mac). Not sure how long these sale prices will be on for, so go get them! I use all of these apps regulary and recommend them often. App store links are affiliate links, so every purchase supports Scripting OS X.

Happy Holidays!

Edovia Screens: iOS, Mac

My favorite VNC/screen sharing application on iOS. Sale is for both Mac and iOS versions.

Byword: iOS, Mac

This is the app I use to write the posts for the newsletter and this blog. It is a simple yet useful markdown editor. Publishing directly from Byword to a weblog site is a free In-App-Purchase. (Not sure if this is part of the sale.)

Duet Display: iOS

Turn your iPad or iPhone into a second (or third) screen for your Mac. Duet Displays requires connection with a USB-Lightning cable to a mac but then you can use that nice retina iPad screen as extra dekstop space.

Junecloud Deliveries: iOS, Mac

My favorite package tracking software. This sale is for the Mac App Store.

Get the “Marketing Name” for a Mac

I am working on making our onboarding/installation process nicer. For that I decided it would be nice for SplashBuddy to say “Welcome to your ’MacBook (Air|Pro)/iMac/Mac (mini|Pro)”.

You can get the Mac model name easily enough from System Profiler:

$ system_profiler SPHardwareDataType
Hardware:
    Hardware Overview:
      Model Name: MacBook Pro
      Model Identifier: MacBookPro14,3

However, this is only the “model family.” When you go into “About this Mac” it give you a more detailed “name” like “MacBook Pro (15-inch, 2017).”

Friendly people on the MacAdmins Slack pointed me towards a property list file in the ServerInformation framework:

/System/Library/PrivateFrameworks/ServerInformation.framework/Resources/English.lproj/SIMachineAttributes.plist

This contains the “Marketing Model Name” among other information for each type of Mac, sorted by the model identifier, which you can also get from system_profiler or with sysctl -n hw.model. Even better this plist is localized, so you can get the marketing name in the current user’s preferred language.

The easiest way to get the correct localization is to use the system’s NSBundle class and methods and the read the property list. I chose Python to do this, since it offers access to NSBundle with the Cocoa-bridge and reading and working with property lists is very easy. (Learn more about working with property lists in Python in my book: ‘Property Lists, Preferences and Profiles for Apple Administrators’)

You can run this script with a parameter:

$ ./modelinfo.py MacBookAir7,2
13" MacBook Air (Early 2015)

When you don’t give a parameter, it will use sysctl -n hw.model to determine the current Mac’s model identifier and use that:

$ ./modelinfo.py
15" MacBook Pro with Thunderbolt 3 and Touch ID (Mid 2017)

Since the script uses NSBundle and does not go directly to a specific localization of the property list file, it will give different language results depending on the preferred language of the user:

$ ./modelinfo.py     # (Dutch)
15-inch MacBook Pro met Thunderbolt 3 en Touch ID (medio 2017)

$ ./modelinfo.py     # (Japanese)
15インチMacBook Pro、Touch Barを搭載(Mid 2017)

Imaging is Dead… Long Live the Installer!

The writing has been on the wall for a long time. With the release of macOS High Sierra, Apple has finally confirmed that imaging is dead.

Apple doesn’t recommend or support monolithic system imaging for macOS upgrades.

I have written a book which expands on this topic and is regularly updated. Please check it out: “macOS Installation for Apple Administrators

The Final Nail

The final nail in the coffin for imaging is this support article: Upgrade macOS on a Mac at your institution

It states the limitations to installing and upgrading macOS with High Sierra:

  • the Mac being installed or updated must be connected to the internet
  • installations and updates cannot be done on external devices, like those connected via Target Disk Mode, Thunderbolt, USB, or Firewire
  • there are four supported methods of installing macOS High Sierra

These methods are also re-iterated in this section in the macOS Deployment Reference

Some of the features have changed since this article was written. You can find an updated post for macOS 10.13.4 here.

Why?

Apple’s stated reason for requiring the installer is to ensure that a Mac’s firmware is all up to date and matches the OS installed on it.

Only the macOS Installer can download and install the firmware update. Firmware updates can’t be done on external devices, like those connected via Target Disk Mode, Thunderbolt, USB, or Firewire.

This is especially important in High Sierra, because to boot into a system on an APFS formatted (or converted disk) the Mac’s firmware needs to be able to mount and read APFS. The firmware that was installed with 10.12 or earlier is not able to read APFS volumes and. When you image a Mac with High Sierra and APFS without updating the firmware, you will get the question mark at boot, because the firmware cannot find a system.

However, the EFI, which manages (among other things) the boot process is not the only “firmware” that needs to be managed on your Mac. Many of the hardware components in your Mac, such as the SSD, the power controller (SMC) and the TouchBar controller on the new MacBooks Pro, have their own firmware that needs to be installed and updated.

Apple has been increasing the protection of the vital parts of the system and hardware. The firmware in these components cannot just be changed by any process. Only the Apple macOS Installer application has sufficient privileges and entitlements to perform these updates. The installer process has to run on the Mac itself, it cannot run over target disk mode.

Future Mac hardware might introduce even more components that require firmware.

On iOS, a secure boot chain prevents tampering with the system on a device after it has been installed. The secure boot chain also prevents replacing the system with an image from another device.

It is conceivable that Apple wants to implement a secure boot system on future Macs as well. Current Macs probably do not have the hardware required to implement this. (The TouchBar MacBooks Pro have a Secure Enclave chip like the iPhone and iPad and might already have the necessary pieces in place.)

So now what?

This has been coming for a long time. Even though APFS is not, as originally predicted, the direct culprit. It is still end-of-the-line for imaging.

However, the news is not entirely dire. Apple has been surprisingly forthright about the direction they want to go. While the documentation is a bit lacking, there are instructions on what the solutions for Mac System Adminstrators should be.

The four supported means of installing and upgrading macOS and the firmware for Macs are a clear direction of what needs to be done. However, Apple is a bit shy on how administrators can and should implement them.

There are a few options:

Put the Burden on the Users

This will work in some deployments where users are in control of their Macs. Either a full BYOD (Bring Your Own Device) scenario, or one where devices provided by the organisation are in full control of the user.

Even when the devices are enrolled in an MDM, users are still administrators and in control. Administrators can use reporting tools, to determine which Macs are capable of installing High Sierra and not upgraded yet.

You can even use reporting tools to gather information on the firmware and whether it matches the latest version.

You can then instruct users with email or notifications to download the High Sierra installer and initiate the upgrade themselves. You should warn them to have a current backup and that the process might take some time, so it should be run overnight.

If you have a software management system in place, you can use that to load the macOS Installer application on the clients and notify the user when it is ready.

The Mac App Store only downloads a “stub” installer application which is then filled in with an extra download. If you have blocked access to Apple Software Update Servers or redirected clients to a local, managed Software Update Server, clients might not get the complete installer application. Greg Neagle has a great post on this.

To save download time for the users, you can also provide USB/Thunderbolt drives with a bootable installer drive. This might speed things up a bit, though it does not really change the process.

How do we Automate this?

As system administrators we want to automate the process, so that it ideally does not require any human interaction. That way we can replicate the process hundreds and thousands of times.

Ideally, we also want to inject some custom steps into the process. Apple provides two means of achieving both of these steps, and some open source tools are providing solutions as well.

NetInstall

A custom NetInstall set built with Apple’s System Image Utility is one of the supported means of installing and updating macOS (and the firmware) to High Sierra.

You can find System Image Utility in /System/Library/CoreService/Applications. You can customize the process and even add your own installer packages, scripts or profiles. Packages used in System Image Utility have to be Distribution Packages.

Since a NetInstall system is like a Recovery system, you can use scripts to control behavior of your Mac like allowed NetBoot IPs, or ‘User-Approved Kernel Extension Loading’ (UAKEL) here as well.

You can even automate the NetInstall process to a point where, once you have chosen the NetInstall volume (when holding the the option key at boot) the remaining process is without interaction. (Though this is a dangerous choice, as it might simply wipe and re-install Macs. Use the other limitation options such as by MAC address or hardware type to keep this safe.)

NetInstall requires Mac running macOS Server. However, BSDPy can replace a NetBoot/NetInstall server and runs on many platforms, including VMs.

Note: as of 10.13.0 there still seem to be a few bugs with NetInstall on HighSierra. It works mostly but seems exceedingly slow. Also some admins have reported problems with adding mulitple packages, profiles or scripts for configuration. 10.13.1 is already in beta and seeding and you should be testing that.

NetInstall on USB

If you do not have the infrastructure to run a NetInstall or BSDPy server, you can also restore the NetInstall.dmg image that System Image Utility creates to an external drive. When mounted on a Mac, it will the High Sierra installer applications, and double-clicking it will start the proper installation process.

Any additional packages, profiles or scripts will be included with this custom external install application as well.

The startosinstall Command

When you already have a management system (Munki, Jamf, Filewave, etc.) you want to initiate the update process with rules or policies. At first glance the supported means of installing macOS seem to be at odds with managed client workflows, since they require user interaction.

However, there is a tool hidden inside the macOS Installer application (since macOS 10.12 Sierra) called startosinstall. The full path to the tool is

/Applications/Install macOS High Sierra.app/Contents/Resources/startosinstall

Note: I believe it was Rich Trouton who first documented this tool in his notes for WWDC 2016. Since then many admins and open source projects have worked to figure out how to use this tool in the best way.

When you run it with the --usage argument you get the following:

$ /Applications/Install\ macOS\ High\ Sierra.app/Contents/Resources/startosinstall --usage
Usage: startosinstall

Arguments
--applicationpath, a path to copy of the OS installer application to start the install with.
--license, prints the user license agreement only.
--agreetolicense, agree to license the license you printed with --license.
--rebootdelay, how long to delay the reboot at the end of preparing. This delay is in seconds and has a maximum of 300 (5 minutes).
--pidtosignal, Specify a PID to which to send SIGUSR1 upon completion of the prepare phase. To bypass "rebootdelay" send SIGUSR1 back to startosinstall.
--converttoapfs, specify either YES or NO on if you wish to convert to APFS.
--installpackage, the path of a package to install after the OS installation is complete; this option can be specified multiple times.
--usage, prints this message.

Example: startosinstall --converttoapfs YES 

There is also an undocumented --nointeraction flag which can be used to run the tool without any user interaction. This is obviously useful for management systems.

Once you have used to your management system to make sure the macOS Installer application is on the client system, you can execute a script with the startosinstall command to initiate the installation process. Remember that the installation process can take a long time, so it should be initiated by the user in a Self Management portal or run during off-hours for kiosk like Macs in labs or classrooms.

startosinstall --applicationpath /Applications/Install\ macOS\ High\ Sierra.app \
    --agreetolicense \
    --nointeraction

The --converttoapfs [YES|NO] argument allows you to suppress automatic APFS conversion on SSD Macs.

There is also a --volume argument to target the non-boot volume. However, this will only work when SIP is disabled or when you run startosinstall from a Recovery/NetInstall disk.

The --installpackage option allows you to add one or more custom packages that will be installed after the OS installation is complete. This is very useful for customization and cleanup. Packages used with startosinstall --installpackage also have to be Distribution Packages.

Note: even though the usage states that you can repeat the --installpackage argument, as of 10.13.0 only the first package given will run the installation with fail with more than one package. Make that one package count. (Note: edited this paragraph. Thanks to Greg for clarifying.)

Tool Support

Is Imaging completely dead?

The imaging tools (like Disk Utility, hdiutil and asr) will work with APFS volumes. However, the support article states:

You can use system images to re-install the existing operating system on a Mac.

So when you need a workflow that requires quick re-imaging, you can use one of the supported methods to install or update the Mac (Firmware and OS) and then use monolithic (or thin) imaging over network or thunderbolt for fast restores. This is useful for scenarios where fast imaging turnaround is required, such as classrooms and labs or loaner laptop setups. However, you have to use extra care to make sure the image system version matches the version that was installed.

Going forward, I expect imaging to be less and less feasible as future Mac hardware and security features will make it harder and harder to use. The times where we could have just one image which will run on all supported Macs might be over as hardware (and the software required to run the hardware) becomes more and more fractured. Note that there is not a unified single ‘iOS’ image/installer for all iOS devices.

Summary

macOS High Sierra 10.13.0 works well for individual users. However, there still are quite a few issues that are relevant for managed deployments. There are many problems with Active Directory and Filevault, NetInstall is slow and adding multiple packages to an installation is broken. Just to mention a few.

Even though it makes sense for some deployments to hold back from High Sierra right now, you will want or have to upgrade soon.

Apple said at WWDC the iMac Pro will ship in December 2017. Its tech specs page, states it will run High Sierra. You can expect it to require High Sierra.

Also, critical security patches might only be pushed for High Sierra.

Imaging is dead. In an unusual move Apple has come right out and said it loud and clear. If you have not done so already, start testing and implementing one of the above strategies right now, so you are ready to move to High Sierra.

Read about more changes with the macOS 10.13.4 updates here.

I have written a book which expands on this topic and is regularly updated. Please check it out: “macOS Installation for Apple Administrators

On Distribution Packages

Distribution packages are a special format of installer packages. Distribution packages can contain one or more normal or component packages. They can also contain extra resources to customize and control the user interface in the Installer application.

In most cases administrators prefer component packages since they are easier to create and maintain. However, there are a few cases where distribution packages are necessary:

  • add a package to a custom installation in NetInstall, AutoDMG or a system installer created with createOSXinstallPkg
  • combine multiple component pkgs into a single installer
  • restrict hardware and system requirements
  • modify the interface presented in Installer.app
  • push the package with MDM’s InstallApplication command

Building Distribution Packages

You can easily convert an existing component package, built with pkgbuild to a distribution package with the productbuild command:

$ productbuild --package component.pkg dist.pkg

You can also combine multiple components into a single distribution package:

$ productbuild --package A.pkg --package B.pkg combined.pkg

You can add the --sign option to the productbuild command when the distribution package needs to be signed:

$ productbuild --sign "Installer: Armin" --package component.pkg dist.pkg

You can find valid identities with

$ security find-identity -p basic -v

The string you pass with the --sign parameter can be a partial match to the full identity name.

Note: munkipkg has a flag to build a distribitution package instead of a component package.

Extracting Component Installers from Distribution Packages

Sometimes you may want to extract a component installer pkg from a distribution package.

First you need to expand the distribution pkg with pkgutil:

$ pkgutil --expand dist.pkg dist_expanded

When you use the --expand option on a distribution package, components will also be expanded into subfolders that end in .pkg. Because of this Finder will erroneously display them as installer bundle files. This is misleading, since the components are not functional in this form.

When you want to use the component package without any modifications, you can quickly recompress or ‘flatten’ the expanded component:

$ pkgutil --flatten dist_expanded/component.pkg component.pkg

The process of expanding and flattening a component will of course remove any signature the original pkg might have had. You can re-sign the flattened package with productsign:

$ productsign --sign "Installer: Armin" component.pkg component_signed.pkg

Note: Obviously, when you are tearing a distribution package apart you need to know what you are doing. Components in a distribution package may depend on other components or on scripts and tools in other components. As always: test, test, test.

Packaging Book

You can learn more on building installer packages in my book: “Packaging for Apple Administrators”

Post WWDC Summary

Earlier this year I wrote a post on whether packaging is dead. Since I wrote a book on Packaging and have also invested much of my career in macOS I do have quite some interest in the topic.

(Please buy the book! If you have bought and read it, please leave a review!)

After that post I made myself a reminder to re-visit the topic post WWDC. I was very much expecting to be proven wrong or hopelessly optimistic. This reminder has been bugging me for a while. I have had a hard time to consolidate my thoughts into writing.

It’s not that this year’s WWDC was boring. Quite the opposite. The new iPads Pro look wonderful and I want one. Apple also announced great new iMacs and MacBooks and a space-grey iMac Pro, demonstrating they still care about the Mac line. (The Mac mini, however, got no love this time around. I do hope the line gets at least a speed bump and we don’t have to wait for a the new Mac Pro to get a decent option for screenless Macs. I’ve given up on servers…) And finally, both iOS 11 and macOS High Sierra (10.13) look like solid updates with lots of new features for users and developers. This was a great WWDC!

Mac admins were concerned that this update would lock down macOS in a similar fashion to iOS. The worst case scenarios painted a picture where not even admin users would be able to get root privileges and you couldn’t install third party daemons and agents any more, fundamentally breaking the way all management systems work. Admins would have to re-work their workflows to work through MDMs, which are not yet capable to bear this burden. The new Apple File System APFS would break NetBoot and all the tools admins use to image Macs.

What happened was… well… nothing much really.

Mac-narök has been postponed.

(Excellent talk by Micheal Lynn at MacDevOps YVR, just a few minutes before the WWDC Keynote. Go watch it.)

There will be changes in High Sierra that affect admins. APFS on macOS is definitely going to happen. In the current (first) beta there is an option to disable the filesystem conversion during upgrade, but it is unknown wether that option will still exist in the release. You can now add iOS devices to DEP even if they were not registered at purchase. You can control a firmware password on Macs with profiles. There are some (minor) changes to files and folders protected by SIP.

I don’t believe or want to suggest the posts above and many other people who predicted the end of Mac Administration as we know it were hysterical or unnecessarily panicked. When they were written there were strong indications and hints that Apple was planning a lockdown of some form soon. MDM only Mac administration might still happen in a future update. However, we seem to have gotten an reprieve, which is good.

Why did the lockdown not happen now? Excellent question, which I do not know the answer to. There was a big outcry from the Mac admin community and many used their official channels (Apple reps and support, Radar, Feedback) to tell Apple what a huge imposition such a quick and drastic change would be. Also many third-party application developers are reluctant to (or cannot) move to the Mac App Store, which would be a requirement in an MDM only world.

For now it seems that common admin tools will run on High Sierra and APFS with just some minor adjustments. This includes packages! Packaging is not dead! Long live Packaging! …and all the other tools!

(On the other hand, some things may still break or be removed during the beta phase.)

Does that mean we should just happily keep doing what we are doing? No. Even if Apple does not yet enforce ‘MDM-only’ they are clearly moving towards ‘more MDM.’ We still have to re-evaluate every setting and workflow with MDM in mind. There are some great solutions already that can combine MDM with e.g. Munki, Chef or Puppet.

Even though imaging, whether you choose the “thick” or “thin” approach, will probably still work in High Sierra, you should be thinking about alternative strategies. DEP plus application installs and updates are more flexible and powerful than full disk imaging.

There are certain setups, such as classrooms and training centers, which require frequent re-imaging with short turnaround times. Ironically, the tech that was predicted to kill imaging might provide a solution. APFS disk snapshots could provide a solution for fast system restores. The tools for this do not seem to be fully in place yet, but the time to test and file bugs is now.

The MDM ‘InstallApplication’ command, which installs the agent software, such as the Munki or Jamf client, should be supported by management systems. This would allow clients to be connected to the management system without user interference and the client software to add to the limited functionality of MDM with tools that admins already have solutions and expertise for.

So the post WWDC summary: the ‘End of Things as We Know Them’ has been postponed. Imaging will still work, but you want to start examining and testing alternatives. Packages and scripts remain relevant, but there are interesting new means of distributing them.

It is already apparent the next WWDC will have more exciting news ready for Mac and iOS Admins. Until then we will be busy learning the new features and tools in High Sierra and iOS 11 and laying the groundwork to the future.