The book should be useful for anyone who uses macOS Terminal and is wondering what the change of the default shell in Catalina means and how to best handle the transition. The book describes the motivation for Apple (and the user) to “move to zsh” and how to get the most out of the new shell.
The blog series added up to about 11K words, and the book, in its current form, is more than 22K words. Compared to the series, I have added images, movies, clarifications, more examples, and several new sections and appendices.
This books explains:
– why Apple is changing the shell
– implications for infrequent and expert Terminal users
– how to move from bash to zsh
– configuring zsh to be more productive
– moving scripts from bash to zsh
And this will certainly not be the end for “Moving to zsh.” Like my other books, I plan to update and add to it after release as well, keeping it relevant and useful.
This is my first book that is not targeted mainly at MacAdmins. I believe this book will be useful for any Mac user that uses the Terminal frequently: Developers, web admins, scientists, and other power users. Please help spread the news by sharing this post and the book link with friends, co-workers, and across social media. Thank you!
It contains lots of updates regarding Catalina, and the usual list of typos and other fixes.
As usual, the update is free when you already own the book.
If you have already purchased the book, you can go to Apple Books application on your Mac and choose ‘Check for available Downloads…’ from the ‘Store’ menu. I have seen the Mac Books app be really slow (or even completely blind) in picking up updates, you can accelerate the process by removing the local download and re-downloading the book. In iOS tap on your iCloud account icon next to ‘Reading Now’ and then choose ‘Updates.’
If you have not yet purchased the book, I have good news for you: I have lowered the price!
The format of the book had the original intention to help MacAdmins learn about and deal with the strange, new post-imaging world that came with the High Sierra and T2 Security chip. I like to believe it did that quite well. But since then, the releases of Mojave and Catalina have added more layers of complexity and information on top of that.
The post-imaging world isn’t new anymore. It is still strange, complicated, and sometimes hard to navigate. However, I feel that the book’s format would have to change to keep being a useful guide. Obviously, such a re-structuring is a massive effort and would pretty much result in a new book. Maintaining and updating a book is a lot of effort, re-writing it even more so.
Thus the decision that this might be last update for “macOS Installation.” Depending on how disruptive the changes in the Catalina “spring” update will be, I might update for those, but I am not planning to update the book for 10.16 next year.
I might work on some new book on macOS deployment and management in the future. However, I have a few other topics I want to publish before I do that, so that might be a while.
Charles Edge’s and Rich Trouton’s new book should be a great successor to “macOS Installation”:
Rich Trouton’s and Charles Edge’s “Apple Device Management: A Unified Theory of Managing Macs, iPads, iPhones, and AppleTVs”: pre-order on Amazon US, UK, DE (Affiliate Links)
“macOS Installation” should remain useful for the life time of Catalina, which, depending on your deployment practices should be another one to four years, more if 10.16 and 10.17 do not drastically change everything again.
Readers who bought the book 16 months ago got several updates for free. I believe free updates are one of the great value propositions of self-published digital books. Most computer related information changes quickly these days and being able to update digital books is a great way to extend their lifetime, usefulness, and value.
My plan to not further update for “macOS Installation” thus lowers its value a bit, and to reflect that I am lowering its price in the store.
That said, I am convinced the book is still very helpful and full of useful information as it is, so if you have not bought the book yet, this is your chance!
You might have seen this coming. My next book will be called “Moving to zsh” and will cover the new default shell on macOS and how to switch to it from bash.
It is based on the series of blog posts that I posted from June through August, but reworked and expanded with more detail and more topics. Like my other books, I plan to update and add to it after release as well, keeping it relevant and useful.
There is a lot of interest on the topic since the release of Catalina and I wanted to let everyone know, that after the blog series, a hands-on training class, and my presentation at MacSysAdmin, I still have more to add to the topic. I set the expected release date to December 31, 2019. “Before the end of the year.” (Like the Mac Pro.) I have hope that it will be done sooner than that, but we will see.
When I have more information, you will, as always, read about it here.
It contains lots of fixes, some new parts and updates with regards to macOS Catalina.
This book is now nearly three years old and if you bought it at the very beginning you have gotten eight updates for free!
(Historic sidenote: v1.1 was just a quick fix to remove some placeholder text, so that was the first version on the iBooks Store.)
If you have already purchased the book, you can go to Apple Books application on your Mac and choose ‘Check for available Downloads…’ from the ‘Store’ menu. In iOS tap on your iCloud account icon next to ‘Reading Now’ and then choose ‘Updates.’
Changes in this version (you can also find this in the book in the ‘Version History’ section):
added a note on the spkg command line tool for Suspicious Package
updated the list of Considerations for Installation Scripts with regards to packages used in Recovery and zsh
updated script code across various scripts to match my updated coding standards
added a note on zsh in About this Book
changed the sample script in the Payload-Free Packages section to enable Screen Sharing instead of SSH because of changes in macOS Catalina security
added information on Notarization to Packages and Gatekeeper
added a note on the new Catalina read-only system volume in Testing the Package
If you have already purchased the book, you can get the update in the Apple Books application on your iOS device or Mac.
While I was putting this book together, macOS High Sierra was very much a moving target and I had to update and change sections frequently, sometimes while they were being written. Thankfully, the changes to macOS deployment and installation have calmed down somewhat since then. While Mojave and the new Mac hardware have brought some changes, they are not as disruptive as High Sierra, UAMDM and the iMac Pro were.
Originally, I made a commitment to update the book up to the macOS Mojave release. This is the book’s first post-Mojave update. I am planning at least one more update to cover any changes the Mojave “Spring Update” might bring.
Once we know more about macOS 10.15 after WWDC, I will decide on the book’s further future. As I have outlined in my 2018 review/2019 outlook post, I don’t expect dramatic changes this year, so it will probably make sense to keep the book going for the lifetime of 10.15.
It is quite likely that the deployment workflows outlined in “macOS Installation” will serve MacAdmins well for the foreseeable future.
This update received quite a bit of new content. I have rewritten and expanded many sections. The book is now eleven pages longer than the third version and sixteen pages longer than the first release. I have added many more links to external pages, tools, posts and support articles where they are available.
The update is free when you have already purchased the book. Unfortunately, the “updates” area seems to have gotten lost in the Apple Books app re-deisgn. You can just follow the link to the Apple Book store and the app should tell you that an update is available.
If you don’t have it yet, buy it now and get all future updates for free!
Here is a detailed list of most of the changes, which you can also find in the ‘Version History’ section of the book itself. (The version history in the book is linked to the actual changed section for easier reference.)
added description of the new –downloadassets option when creating external installer drives
updated description the –preservecontainer option in Erase Installation
updated descriptions of Secure Boot in various locations to reflect the Mac models introduced in October 2018
added an overview table for which current Mac models have Secure Boot and their minimum OS Versions
added links to the support pages for more versions of macOS in Older macOS Versions
moved and extended the section on Hardware Specific Systems and Installers
added downgrading limitation to Erase Installation
added description of fdesetup list to the FileVault section
added description of Twocanoe’s MacDeployStick to Sideloading Packages
added Manual Enrollment by IT to Manual Enrollment
added EraseInstall application to Restoring macOS
added link to support article with supporting servers to Device Enrollment Program
While you are all watching the progress bars downloading macOS Mojave, I have submitted updates to two of my books!
(Remember to archive the macOS Mojave installer app before updating!)
The service-formerly-known-as-iBooks is now called “Apple Books” or just “Books” across all Apple platforms. Whatever the name, my books are available there.
Both “Packaging” and “macOS Installation” have received updates with several new passages regarding macOS Mojave and other fixes and rewrites.
For “Packaging” I have also started the process of re-visiting all the examples. You never stop learning and I have certainly learned a lot in the last two years. Some of the scripting and writing styles I used when I started writing the book nearly three years ago now seem odd to me. This rewrite is a major process and so far I got to chapter 2. This remains a work in progress and expect more updates here soon. (All updates, as usual will remain free for those who have purchased the book.)
“macOS Installation” got less changes and updates than I expected for the Mojave update. We have been testing Mojave internally and the strategies for deploying High Sierra still work for Mojave. With the new APFS support for all drive hardware admins should be able to simplify deployment workflows. Nevertheless, I am sure that much more will be revealed over the next few weeks, now that the NDA is lifted and when the first update(s) drop(s). I will keep updating this book also.
The content of my third book “Property Lists, Preferences and Profiles” (PR3) has not really been affected but the Mojave release. I am also working on an update for this book with some new content added. When it happens you will learn about it here.
All of that said, today I pushed an update for my other two books to the iBooks Store! The main motivation was to update the “More Books” section in both with a link to the new book. But along with came a bunch of small fixes, corrections and even new subsections that have accumulated since the last update. (These are the eighth update for “Packaging” and the third update for “PR3”) You can see the details in the version history section in the iBooks Store and in each book.
If you have already bought the books, just go to your iBooks app and download the updated versions. If you have not bought them yet, you can get them here:
There is one more thought I want to add: I don’t consider this book complete yet.
This doesn’t sound like the best endorsement, so let me explain:
Apple is not only deprecating technologies for macOS deployments (imaging, NetBoot), but the pace for macOS releases is changing as well. I started writing this book in October, shortly after the release of High Sierra. However, with every update to 10.13 it became clear that Apple was not standing still. They kept iterating, sometimes quite drastically.
The iMac Pro introduced Secure Boot and we realized that NetBoot-based workflows would need to be retired much faster than initially assumed. Secure Boot also makes booting off external drives quite inconvenient. While this is great from a security standpoint, it removes yet another option for installation workflows.
Not even the proofreaders have seen the final version. (Can’t blame the remaining typos on them.) Existing workflows (or planned workflows) had to be adapted for the new hardware and updates.
I don’t expect this will stop at 10.13.4, or 10.13.5 or any time soon. We will have to update our practices to be more adaptable, and that includes writing books. I expect that 10.13.5 and future updates, new hardware with new features, and then 10.14 (whatever Apple will call it) will bring more changes.
Digital books are software, and can be treated as such. I wanted to “release early and often.”
When you buy the book now, you will receive new versions from the iBooks Store when I publish updates with new information, workflows and solutions. I plan to keep this up with the remaining 10.13 updates and all the way to the 10.14 release (presumably this fall). So by the end of this year, you should have a book with great information on how to install High Sierra and how to upgrade to and install 10.14.
Like for an app in the App Store, I have to consider how long it makes fiscal sense to maintain free updates. After the 10.14 release, I will evaluate the effort spent and make new plans from there. I will certainly write about that here.
There are also some parts of the book that I would have liked to have a bit more detail. I had set the myself the goal to get the book out before WWDC and 10.13.5, so some things had to put on the ‘later’ list. So, in addition to new content forced on us by updates and upgrades, I will keep improving, fixing, and adding to useful content as well.
I believe that “macOS Installation for Apple Administrators,” as it is now, already has a lot of great information and is well worth buying and reading. It will get even better.
At the end, a short request for help: The book is self-published. There is no big marketing machine behind this. If you like the book, then please leave a review on the iBooks Store. Apple’s stores segregate reviews by region, so every single review makes a big difference.
You can also help by recommending the book to another MacAdmin or presenting it at your local MacAdmins meeting, or just by simply sharing or re-tweeting my posts on social media.
While APFS has certainly come with some challenges, mainly for managed FileVault users, it was not the “Macnarök” that we were expecting.
I am not including these links to make fun of the authors – quite the opposite. At the time there was justified anxiety about the future of macOS adminstration and these posts (and many others) put this into words. I believe that because of MacAdmins blogging and presenting about their concerns and anxiety, talking with Apple Reps and filing bugs, as a community, were able to influence the course Apple is taking. It is definitely a slow, frustrating and opaque process, but not entirely fruitless.
However, other new “features” of High Sierra such as UAKEL and, later with 10.13.4, UAMDM provide their own set of hurdles administrators have to clear.
These are the main challenges for macOS deployment:
Imaging is Dead, Long Live the Installer
MDM is Essential
Let’s look at these challenges and some solutions to them in detail.
Imaging is Dead, Long Live the Installer
Imaging has been the means to quickly and reliably install macOS to clients. It could scale from an external drive for a few clients to a NetBoot infrastructure for large deployments.
In the early days imaging meant building a ‘Golden Master’ image with all configuration and software ‘baked in,’ which then would be deployed to all the Macs. Creating this ‘Golden Master’ image was a lot of work and the result quite fragile and error-prone. Macs would usually be kept on a specific version of ‘the image’ for months or a year without updates.
Since the process of manually building the Golden Master was tedious and error-prone, administrators started to automate the process. Tool chains of packages, scripts and other tools could build the master image (or most of it) automatedly.
InstaDMG was one of the first tools to do this. AutoDMG is a direct successor of this approach and still useful for many tasks today.
The same packages and scripts that automated the configuration of the master image could be used to install and update software on ‘live’ Macs. Management tools, like Munki, Casper (now Jamf Pro) and Filewave, can deploy updates to clients automatically without user interaction. As work was moved from preparing the master image to maintaining the live system, the master image became thinner and thinner.
This resulted in the ‘thin imaging’ approach, where the image only had the minimal ‘base OS’ plus enough configuration to connect it to the management system. Once the Mac booted the management system takes over. Imaging would only be used to wipe a Mac and lay down a ‘base image.’
While this approach is slower than monolithic imaging, it is more flexible and allows to keep the system and software on the client continually updated, without having to re-image.
Some workflows even skipped the imaging entirely. When you get a Mac out of the box it already has macOS installed, all you need is to configure it to connect to your management system, which will take care of the rest.
Some administrators had already moved to ‘thin’ imaging or ‘no-imaging’ workflows before High Sierra and pronouncements that ‘Imaging is Dead!’ have been made years ago. Administrators using thin imaging or no-imaging workflows will find they are better prepared for the High Sierra installation workflows.
Mac administrators now have to build new workflows based on the Apple’s supported installation methods for High Sierra. These all rely on the macOS Installer application in one form or another.
The supported means of installing macOS High Sierra are:
the macOS Installer application
a bootable installer on an external drive
the macOS Recovery System
a NetInstall image built with Apple’s System Image Utility
Using the macOS installer application to upgrade a system from 10.12 is straightforward enough and similar to what was possible with the startosinstall command since OS X El Capitan 10.11.
The main challenge are ‘nuke-n pave’ workflows, where the system on a Mac is erased and overwritten with a new system.
This workflow is easy with traditional imaging. In fact, it was easier and faster to just image over a system, rather than upgrade.
The startosinstall --eraseinstall option added in 10.13.4 makes an installer based ‘nuke’n pave’ workflow possible with the macOS Installer application.
In the new High Sierra world of Mac deployment, we face the same challenge: how do you connect an ‘out-of-the-box’ macOS to your management system.
Apple’s answer to that is clearly DEP.
MDM is Essential
DEP will tell a freshly installed Mac which MDM server is ‘responsible’ for it. The MDM server can then use the InstallApplication MDM command to install and configure a management agent, which together with the mdmclient performs the remaining configuration and software installation.
Up until 10.13 Mac administrators could ignore and not use an MDM for Mac deployment and management. There were a few useful features, but there are also many tasks an MDM cannot do or that a local agent can do better.
With High Sierra MDM became essential to manage and support UAKEL. In the early versions of High Sierra, merely being connected to an MDM would disable UAKEL, starting with 10.13.4 UAKEL can be managed with a configuration profile which has to be pushed from a user approved MDM.
Like supervison for iOS devices, UAMDM adds an extra level of approval and provides an additional level of control over the macOS client device. UAMDM will be required for more and more essential functionality going forward.
However, as the name implies, a ‘user approved’ MDM needs to be interactively approved by a user at some time during deployment which creates an enormous challenge for automated workflows.
Apple claims that DEP enrolled devices are automatically user-approved. However, even the DEP enrollment still requires user interaction in the Setup Assistant.
Also DEP is not available in all regions and it may not be possible for organizations to add all existing Macs to their DEP account. DEP also requires Apple DEP servers to be accessible. So during this transition phase and for some other situations, Mac administrators have to have a non-DEP workflow solution to get a UAMDM enrolled device.
Finally the iMac Pro introduces a new system controller with secure boot. By default all external means of booting an iMac Pro are disabled.
NetBoot/NetInstall is not supported on the iMac Pro at all. You can enable booting off external drives, but the process to do so is quite convoluted and cannot be automated. So for administrators, the option is fairly useless.
With the startosinstall command administrators can achieve the necessary management steps from within the local system. You can either upgrade the existing system or ‘nuke’n pave’ with the eraseinstall option.
However, the eraseinstall option is limited to systems that are already on 10.13 with an APFS file system. On the iMac Pro you can safely make that assumption, but for ‘older’ Macs and systems you usually cannot.
So administrators need a second workflow for systems and hardware that cannot run the eraseinstall option (yet).
The Recovery system, external installer volume or NetInstall are the Apple Supported means which can ‘nuke’n pave’ a Mac. NetInstall is the only means that can be automated.
However, there are other solutions such as Imagr or a modified DeployStudio workflow which can achieve the same goal using the macOS Installer application and startosinstall.
The New macOS Deployment Workflow
Deployment strategies and workflows are as varied and individual as the organizations that use them. But with the challenges listed above, I believe we can state a few pieces that are necessary for a new deployment workflow for High Sierra and hopefully going forward.
Enrolling a ‘new’ Mac
First, any deployment workflow should be designed to start with a clean ‘out-of-box’ macOS. This way the same workflow can be applied to new Macs and to Macs that were just re-installed.
At the end of the installation workflow the Mac is enrolled and user-approved with your management system, so that it can take over and perform the remaining configuration and installation. All configuration steps should be controlled from the management system and be executed independently of which deployment workflow was used.
DEP is the Apple-preferred means of connecting a new device to your MDM. Even when you require a non-DEP workflow for older Macs you should implement a DEP workflow. It is likely that Apple may make DEP enrollment mandatory in the future.
However, right now in the transition phase, you will need a second workflow for machines that cannot be registered with DEP.
For the non-DEP workflow, you can choose between manual enrollment, which happens after the user has booted and logged in for first time, or some sort of ‘side-loading’ where you add the MDM and management system configuration to a system, before it even boots for the first time.
For sideloading you can boot to Recovery and install software on the system volume. (Bootstrappr)
On Macs that still support NetBoot, you can also use tools like NetInstall, Imagr or DeployStudio. Either way you should do this before first boot of the system.
UAMDM requires user interaction to approve the management in some form or another. This is a change from previous deployment workflows which could be ‘fully automated.’
You will have to plan for this user interaction. In some cases, such as individual-use devices, the approval of the management will be part of standard first setup and will be done by the end user. In other situations such as classroom setups, it will be harder to make this part of the workflow.
Some management systems let you scope software installations and configurations depending on criteria reported from the device. In this case you can set up your workflow so that installations that rely on UAMDM (third party kernel extensions) are deferred until the MDM is approved.
Either way, the end-result of each of these paths should be a Mac that is enrolled (user-approved) with your MDM where the management system is installing, configuring, maintaining the system and software.
Restoring a Mac to ‘new’
You will also need a second set of workflows that bring a Mac from any state (even a broken one) back to the ‘out-of-box’ state so that you can start over (or retire it).
The first option uses the startosinstall --eraseinstall command. This is easiest to automate and works on new hardware like the iMac Pro. However, it will not work on Macs without APFS systems and older versions of macOS.
The Macs that cannot use the eraseinstall option, however, can use NetInstall or a similar NetBoot based tool. You can use the macOS Install application and startosinstall with Imagr and DeployStudio, so this combination will run the installer and can make sure the firmware is up to date as well.
Note that the workflows and tools for restoring a Mac are also used to ‘sideload’ the agent configuration on to a Mac. You can add the MDM/agent configuration to the restore workflow to save some steps here. However, since NetBoot based workflows are transition solutions to support ‘old’ Macs and systems, your deployment workflows need to work with ‘out-of-box’ systems as well as pre-configured systems.
In the transition phase until all hardware supports eraseinstall you will need both workflows available.
You should also have documentation, training, and support articles to instruct administrators, techs, and users to restore a Mac to ‘out-of-box’ state with (Internet) Recovery or an external boot drive. While this should be an option of last resort, it will be necessary in situations where a Mac cannot boot otherwise.
Obviously, these are very high level views of the workflow and the devil is very much in the details of the implementation. Even a book can only skim the surface of any implementation. Nevertheless, once you understand and accept this framework, the necessary detailed steps become obvious, though not necessarily easy.