Weekly News Summary for Admins — 2019-10-18

Another week, another iOS update. And also another macOS Catalina… Supplemental… Update?

As a Mac admin, I am getting envious of iOS system version numbering.

While I grumble about the nomenclature, the “Supplemental Update” fixed the persisting request to log in to iCloud i had on my MacBook, so it is welcome, whatever it is called.

Note: because of travel and vacation, there will be no Newsletter next week, October 25. The newsletter will be back on November 1!

If you would rather get the weekly newsletter by email, you can subscribe to the Scripting OS X Weekly Newsletter here!! (Same content, delivered to your Inbox once a week.)

On Scripting OS X

News and Opinion

MacAdmins on Twitter

  • mikeymikey: “32-bit Intel Macs: Introduced 13 years ago – stopped shipping that same year. Every since has been 64-bit. 10.7 Lion: Launched 8 years ago in 2011 – required 64-bit. macOS had warnings about 32-bit app launches in 10.13 and 10.14. Why was ANY dev shipping 32-bit -only-?” (Thread)
  • Erik Schwiebert: “It took us /years/ to get rid of the Carbon code in Office and make the switch to 64-bit. And we were done in 2016…”
  • John Goering: “A WWDC slide from a more civilized time in Apple’s history.” (Image)
  • Armin Briegel: “System Image Utility is gone from macOS Catalina. It is not possible to build a NetBoot or NetInstall nbi based on Catalina. (You can probably hack one together, this is not meant as a challenge.) No currently sold Mac model is able to NetBoot. NetBoot is dead.”

macOS Catalina

Bugs and Security

Support and HowTos

Scripting and Automation

Apple Support

Updates and Releases

To Listen

Support

There are no ads on my webpage or this newsletter. If you are enjoying what you are reading here, please spread the word and recommend it to another Mac Admin!

If you want to support me and this website even further, then consider buying one (or all) of my books. It’s like a subscription fee, but you also get a useful book or two extra!

New Book: Moving to zsh

You might have seen this coming. My next book will be called “Moving to zsh” and will cover the new default shell on macOS and how to switch to it from bash.

It is based on the series of blog posts that I posted from June through August, but reworked and expanded with more detail and more topics. Like my other books, I plan to update and add to it after release as well, keeping it relevant and useful.

The book is progressing nicely, but not yet ready. I have put it up for pre-order on the Apple Books Store.

There is a lot of interest on the topic since the release of Catalina and I wanted to let everyone know, that after the blog series, a hands-on training class, and my presentation at MacSysAdmin, I still have more to add to the topic. I set the expected release date to December 31, 2019. “Before the end of the year.” (Like the Mac Pro.) I have hope that it will be done sooner than that, but we will see.

When I have more information, you will, as always, read about it here.

Packaging Book Update: v1.10

I have updated my book “Packaging for Apple Administrators”!

It contains lots of fixes, some new parts and updates with regards to macOS Catalina.

This book is now nearly three years old and if you bought it at the very beginning you have gotten eight updates for free!

(Historic sidenote: v1.1 was just a quick fix to remove some placeholder text, so that was the first version on the iBooks Store.)

If you have already purchased the book, you can go to Apple Books application on your Mac and choose ‘Check for available Downloads…’ from the ‘Store’ menu. In iOS tap on your iCloud account icon next to ‘Reading Now’ and then choose ‘Updates.’

Changes in this version (you can also find this in the book in the ‘Version History’ section):

  • added a note on the spkg command line tool for Suspicious Package
  • updated the list of Considerations for Installation Scripts with regards to packages used in Recovery and zsh
  • updated script code across various scripts to match my updated coding standards
  • added a note on zsh in About this Book
  • changed the sample script in the Payload-Free Packages section to enable Screen Sharing instead of SSH because of changes in macOS Catalina security
  • added information on Notarization to Packages and Gatekeeper
  • added a note on the new Catalina read-only system volume in Testing the Package
  • fixed some mis-spellings and inconsistencies
  • fixed some broken links in Recommended Reading
  • changed to new ‘Apple Books’ nomenclature
  • fixed a dead link in ‘Installation Scripts’

Go get it in the Books store!

Weekly News Summary for Admins — 2019-10-11

Release week! (again)

macOS 10.15 Catalina was released this Monday.

You may be terrified of dialog fatigue or excited about new features like Sidecar. You may be waiting for some critical third party to fix their installers or not planning to upgrade for a year anyway.

Whether you are holding back the update or diving in right away, this means a lot of extra work and learning for MacAdmins. Thanks to all the MacAdmins on Twitter, Slack, and weblogs for helping and sharing information.

If you would rather get the weekly newsletter by email, you can subscribe to the Scripting OS X Weekly Newsletter here!! (Same content, delivered to your Inbox once a week.)

On Scripting OS X

macOS 10.15 Catalina

Adobe and Catalina

MacAdmins on Twitter

  • Jerry Olla: “Need to be able to see Wi-Fi connection details in iOS? Install the “Wi-Fi for iOS” diagnostic profile”
  • Tom Bridge: “Quick PSA based on some feedback from friends: Do NOT notarize someone else’s software, your Developer ID might end up in hot water.”
  • Robert Hammen: “Want to block macOS Catalina from showing up in Software Update preferences on macOS Mojave? sudo /usr/sbin/softwareupdate --ignore "macOS Catalina" prevents it from appearing! Credit to @wegotoeleven”
  • Pepijn Bruienne: “And for those interested in implementing Watch-based auth in macOS 10.15 themselves, take a look at this newly added ACL as part of the Keychain Services API
  • William Smith: “MacAdmins, if you’ve told your technicians to use the Command-Option-R method to boot into Internet Recovery mode on your Macs to reinstall macOS, be aware this will now install Catalina not Mojave.”
  • Erik Schwiebert: “Mac Office 16.29 and later are fully supported on the new macOS 10.15 Catalina! Version 16.31 (due out in mid-November) will drop support for macOS 10.12 Sierra.”
  • Eric Holtam: “Confirmed by @ClassicII_MrMac – a bricked T2 that failed previous DFU bridgeOS restores can be revived.”
  • Jason Broccardo: “Which OSes are offered 10.15 in the GUI Software Update: – 10.10: 10.13 is offered – 10.11: untested – 10.12: 10.15 not offered in SU or App Store>Updates – 10.13: 10.15 not offered in SU or App Store>Updates – 10.14.x: 10.15 offered in SU GUI but not CLI”
  • Rosyna Keller: “I wanted to clarify a few things that have caused some confusion about the Jan 2020 date. 1. All software created after June 1st, 2019 must still be notarized for Catalina. 2. A notarization ticket issued before Jan 2020 still remains valid forever.”
  • John C. Welch: “Yeah, the auth dialogs in Catalina can be annoying. Anyone got a better method besides “pretend your mac isn’t under constant attack and hope for the best” aka “hopes and prayers” or “get cissp certified before you’re allowed to use a computer”?” (Thread)
  • Nikolaj Schlej: “macOS Catalina brings a small, but important update to Mac Firmware Password mechanism: you can now opt-out (and back in) from password reset mechanism used by AppleCare. It makes FW password harder to reset, use it on your own risk!”

Bugs and Security

Support and HowTos

Scripting and Automation

Apple Support

Updates and Releases

To Watch

To Listen

Just for Fun

Support

There are no ads on my webpage or this newsletter. If you are enjoying what you are reading here, please spread the word and recommend it to another Mac Admin!

If you want to support me and this website even further, then consider buying one (or all) of my books. It’s like a subscription fee, but you also get a useful book or two extra!

Notarization for MacAdmins

Apple introduced Notarization in macOS Mojave. Since its introduction Apple has kept increasing the use of notarization checks in macOS. For macOS Catalina, Apple has been very vocal saying that Notarization is a requirement for distribution of Applications outside of the Mac App Store.

This has left many MacAdmins confused and concerned. A large part of the work as a MacAdmin consists of (re-)packaging applications, configuration files and scripts so they can be distributed in an automated fashion through a management system, such as Jamf Pro, Munki, Fleetsmith, etc.

Do MacAdmins need to notarize all the package installers they create as well? Do MacAdmin need to obtain an Apple Developer ID? How should MacAdmins deal with notarized and non-notarized applications and installers from third parties?

This post is an attempt to clarify these topics. It’s complicated and long, bear with me…

Signed Applications

Apple’s operating systems use cryptographic signatures to verify the integrity and source of applications, plug-ins, extensions, and other binaries.

When an application, plug-in, extension, or other binary (from now on: “software”) is signed with a valid Apple Developer certificate, macOS (or iOS, tvOS, and watchOS) can verify that the software has not been changed or otherwise tampered with since it was signed. The signature can verify the source of the signature, i.e the individual Developer account or Developer team whose Developer identity was used to sign the software.

If the contents of the software were changed for some reason, the verification fails. The software can be change by accident or with malicious intent, for example to inject malicious code into an otherwise beneficial piece of software.

Since Apple issues the Developer IDs, they also have the option of revoking and blacklisting certificates. This usually happens when a Developer ID has been abused to distribute malware. The Malware Removal Tool or MRT is the part of the system that will identify and block or remove blacklisted software.

App Store Distribution

Applications distributed through Apple’s App Stores have to be signed with a valid Developer ID. A developer needs to have valid subscription ($99 per year for individuals and $299 for organizations) to obtain a certificate from Apple.

When a developer submits software to an App Store on any Apple system, the software will be reviewed by Apple to confirm whether it meets the various guidelines and rules. This includes a scan for malware.

App Store applications also have to be sandboxed, which means they can only access their own data (inside the “sandbox”) and not affect other applications, services, or files without certain “entitlements” and, in many cases, user approval.

App Store rules and regulations and sandbox limitations preclude many types of applications and utilities. On iOS, tvOS and watchOS, they are the only way for developers to distribute software to end users.

Apple provides a method for Enterprises and Organizations to distribute internal software directly without going through the App Store and App Store review. This should be limited to distribution to employees and members of the organization (such as students of a university or school). This method has infamously been abused by Facebook and other major companies which lead to Apple temporarily revoking their certificates. (We will not discuss Enterprise App Distribution in this post.)

There is also much criticism about how realistic Apple’s rules and guidelines are, how arbitrary the review process is, and whether the sandbox restrictions are useful or unnecessarily draconic. A lot of this criticism is valid, but I will ignore this topic in this post for the sake of simplicity and brevity.

Software downloaded from the App Store is automatically trusted by the system, since it underwent the review and its integrity and source can be verified using the signature. In the rare case that some malicious software was missed but the review process, Apple can revoke the Developer certificate or blacklist the software with the Malware Removal Tool.

Distribution outside of the Mac App Store: Gatekeeper and Quarantine

As mentioned before, iOS, tvOS, and watchOS applications have to distributed to end users through the App Store, be signed with a valid Developer ID and under go the review.

Because the Mac existed a long time before the Mac App Store, software vendors have many ways of distributing software. Originally software was sold and delivered on physical media (Floppy Disks, CDs, and DVDs), but we with the rise of the internet, users could simply download software from the developer’s or vendor’s website or other, sometimes dubious, sources.

Apple has (so far) accepted and acknowledged that these alternative means of software distribution and installation are necessary on macOS. To provide an additional layer of security for the end user in this use case, Apple introduced Gatekeeper in OS X 10.8 Mountain Lion.

When a user downloads a software installer or archive from the internet it is ‘quarantined.’ When the user attempts to install or launch the software for the first time, Gatekeeper will evaluate the software. There are many steps in this evaluation, and Howard Oakley explains the process in much detail in this post.

You can see the quarantine flag with the xattr command:

% xattr ~/Downloads/somefile.pkg
com.apple.macl
com.apple.metadata:kMDItemWhereFrom
com.apple.quarantine

You can delete the quarantine flag with xattr -d com.apple.quarantine path/to/file. Usually, there is no real need to.

The first step of the evaluation is verifying the software’s signature, Developer ID, integrity. When encountering an unsigned piece of software the user will be presented with a warning dialog.

Users with administrator privileges can bypass Gatekeeper by choosing “Open” from the context menu instead of double-clicking to open. Gatekeeper can be completely disabled with the spctl command, though this is not recommended.

The Developer signature provides a way to verify the source and integrity of a piece of software, but since the distribution happens outside of Apple’s control, a malicious developer could still put any form of malicious code in the signed software to keep Gatekeeper happy. As long as the malware avoids widespread detection it will look good to Gatekeeper and the end user. Even when the malware is detected by Apple and the Developer ID is revoked, it is not hard for a malicious developer to obtain or steal a new Developer ID and start over.

Enter Notarization

Apple needed another layer of security which could scan software for known malware and enforce a certain set of security rules on third party software, even when it is distributed outside of the Mac App Store.

Note: I find the effort Apple is putting in to Gatekeeper and Notarization quite encouraging. If Apple wanted to restrict macOS to “App Store only” distribution in the near future, this effort would not be necessary. This shows that Apple still acknowledges the important role that independent software distribution has for macOS.

To notarize software, a developer has to sign it with their Developer ID, and upload it to Apple using Xcode or the altool command. Then Apple notarization workflow will verify that the software fulfills certain code requirements and scans for certain malware. The exact details of what is considered malware are unknown. However, we do know that the process is fully automated and, unlike the App Store approval process, does not involve human reviewers.

If the software has passed the notarization process the result will be stored on Apple’s servers. When Gatekeeper on any Mac verifies the software it can confirm the notarization status from Apple’s servers. Alternatively, a developer can ‘staple’ a ‘ticket’ to the software, which allows Gatekeeper to confirm the notarization status without needing to connect to Apple.

When Gatekeeper encounters a quarantined software that is notarized, it will show the familiar Gatekeeper dialog with an additional note that:

“Apple checked [the software] for malicious software and none was detected.”

Since 10.14.5, When Gatekeeper encounters signed software that is not notarized it will show the standard dialog but with an additional yellow warning sign.

As with the previous Gatekeeper checks for a valid signature an administrator user can override the check by choosing ‘Open’ from the context menu instead of double-clicking to open.

In Mojave notarization was enforced in Gatekeeper checks for kernel extensions and in 10.14.5 for software with new Developer IDs, which where created after June 2019.

Starting with Catalina, all software needs to be notarized to pass Gatekeeper when the first launch or installation is initiated by a user.

However, the warning can still be overridden by an administrator user using the context menu.

What can be Notarized

As of now, the following pieces of software can be notarized:

  • Application bundles
  • Kernel Extensions
  • Installer Packages (pkg), Disk images (dmg) and zip archives

When you are building other types of software, such as command line tools, you can (and should) place them in one of the archive formats. The preferred choice for MacAdmins should be an installer package (pkg) since it will also place the binary in the correct location in the file system with the correct access privileges.

What cannot be Notarized

You should not notarize a binary or application that you did not sign! The Developer ID used to sign a binary (application or command line tool) should be the same as the Developer ID used to submit the software for notarization.

Apple has loosened the requirements for notarization until Jan 2020 to give developers some extra time to adapt. Once the requirements return to the full restrictions an attempt to notarize third party software with a different Developer ID will fail. (Existing notarizations will remain valid after that date.)

Installer command

When you install software using the installer command from the Terminal or a script, it will bypass quarantine and the Gatekeeper check.

This is also true when you install software using a management system such as Jamf Pro, Munki, Fleetsmith, etc.

Software you re-package as a MacAdmin for distribution through management systems does not need to be notarized.

Given this and the limitations on notarizing third party software above, you should very rarely need to notarize as a MacAdmin.

Example: Re-packaging third party software from dmg

A lot of applications for macOS are distributed as disk images. The normal end user workflow would be to mount the dmg after downloading, and then copying the application from the dmg to the /Applications folder.

There are two steps where Gatekeeper might trigger: when you mount the disk image and when you launch the application after copying for the first time. To pass both these checks, a developer should prudently notarize both the disk image and the application. Google Chrome for example does exactly that, avoiding the Gatekeeper warning.

We can verify this with the spctl command:

% spctl -a -vv -t install ~/Downloads/googlechrome.dmg
/Users/armin/Downloads/googlechrome.dmg: accepted
source=Notarized Developer ID
origin=Developer ID Application: Google, Inc. (EQHXZ8M8AV)

% spctl -a -vv -t execute "/Volumes/Google Chrome/Google Chrome.app"
/Volumes/Google Chrome/Google Chrome.app: accepted
source=Notarized Developer ID
origin=Developer ID Application: Google, Inc. (EQHXZ8M8AV)

Unfortunately, some management systems don’t understand “Apps in disk images” as a distribution method. For these systems MacAdmins need to re-package the application into a pkg. You can do that quickly with pkgbuild:

% pkgbuild --component /Volumes/Google\ Chrome/Google\ Chrome.app --install-location /Applications/ GoogleChrome.pkg
pkgbuild: Adding component at /Volumes/Google Chrome/Google Chrome.app
pkgbuild: Wrote package to GoogleChrome.pkg

or use quickpkg.

This new installer package will be neither signed nor notarized:

% spctl -a -vv -t install GoogleChrome.pkg                          
GoogleChrome.pkg: rejected
source=no usable signature

When you send this installer package to another Mac with AirDrop, the receiving system will attach the quarantine flag. And when you double click it, you will get the Gatekeeper warning. However, when you can still install it using the installer command in Terminal, which bypasses the Gatekeeper system, just as your management system will:

% installer -pkg ~/Downloads/GoogleChrome.pkg -tgt /

Alternatively, you can choose “Open” from the context menu in Finder to bypass Gatekeeper. However, this is not something you want to teach your end users to do regularly.

Firefox can be downloaded as a disk image as well as a installer package. While the application inside both is notarized, neither the disk image nor the installer package are. The disk image mounts with no issues, but when you try to open the installer pkg by double-clicking you will get the expected notarization warning. Nevertheless, the pkg will work fine after importing to your management system.

Edge cases

There are some cases where notarization would be useful for MacAdmins but might not even be possible. I met a MacAdmin working at a university at MacSysAdmin last week. They need to re-package a VPN client with customized configuration files to be installed on student-owned machines.

There is really no solution without the students running into the notarization warning. Teaching the users how to bypass Gatekeeper is not a good solution.

In these cases you have to work with the software vendor and Apple to find a workable solution.

Summary

Notarization is a new security layer introduced by Apple in Mojave. The restrictions imposed on non-notarized software increase in Catalina.

When an Application is installed or launched for the first time by the user (by double-clicking) Gatekeeper will verify the signature and notarization status and warn the user if any are missing.

Developers should sign and notarize their applications and tools.

Mac Administrators should not notarize applications and tools from third parties.

Applications and packages installed through management systems bypass Gatekeeper and do not need to be notarized.

Conclusion

Apple is loudly messaging that notarization is absolutely required for applications in Catalina. While this message makes sense for the developers building the software, it does not apply to administrator who re-package third party software for distribution through management systems.

MacAdmins should join Apple in demanding signed and notarized binaries and installer packages from developers.

However, MacAdmins can also continue their current workflows for re-packaging and distribution.

Links

EraseInstall Update 1.2.1

Just a “minor” update. The EraseInstall app now shows the progress that the startosinstall command gives in the command line. This also should help with some better error reporting when the startosinstall command errors out.

I say “minor” but small UI change required some major rewiring underneath. It also required us to dive deeper into how shell commands are executed from Swift than we wanted to.

We have also tested this version to work with macOS Catalina which was released yesterday.

We have more “major” features planned for the future!

You can download the latest EraseInstall Installer and Code here!

Download a full ‘Install macOS’ app with softwareupdate in Catalina

Catalina is out! While you are preparing for your upgrade, here’s a nice new feature for MacAdmins:

The softwareupdate command has gained a new option in Catalina:

% softwareupdate --fetch-full-installer

Will download the latest ‘Install macOS’ application to this Mac’s /Applications folder. This is extremely useful for many admin tasks.

The --fetch-full-installer flag has a sub-flag: --full-installer-version which allows you to download a specific version.

% softwareupdate --fetch-full-installer --full-installer-version 10.14.6

During my testing in the Catalina beta version I was able to download 10.15, 10.14.6, 10.14.5, and 10.13.6. I was not able to test if 10.13.6 would download the hardware specific build of 10.13.6 for the 2018 MacBook Pro, since I do not have that hardware.

I would assume that downloading an Installer application for a macOS version that is not supported on the hardware you are running the command on would fail. (Again I did not have such hardware available for testing.)

So far the only way to download the macOS Installer in some automated fashion was using Greg Neagle’s installinstallmacos.py script. That script still has some abilities that do not seem to be available to the softwareupdate command, but it is good to see Apple accepting the need for this kind of workflow.

Weekly News Summary for Admins — 2019-10-04

Greetings from Gothenburg, where I have been attending MacSysAdmin conference this week. I am humbled to be among the amazing line-up of speakers again. My thanks and congratulations to the organizers, speakers, sponsors, and, especially, attendees for another wonderful conference! See you all next year!

In other news, we got Catalina beta 9 earlier this week and then ‘GM seed’ yesterday. So, this also seems to be coming to a close. Several other Apple related updates as well.

If you would rather get the weekly newsletter by email, you can subscribe to the Scripting OS X Weekly Newsletter here!! (Same content, delivered to your Inbox once a week.)

On Scripting OS X

News and Opinion

MacAdmins on Twitter

macOS Catalina

Bugs and Security

Support and HowTos

Scripting and Automation

Apple Support

Updates and Releases

Support

There are no ads on my webpage or this newsletter. If you are enjoying what you are reading here, please spread the word and recommend it to another Mac Admin!

If you want to support me and this website even further, then consider buying one (or all) of my books. It’s like a subscription fee, but you also get a useful book or two extra!