Category: Packaging

Update: quickpkg 1.0beta

QuickPkg is one of those tools that I built for myself a long time ago and has remained useful (at least to me) without many updates.

Since Apple will be removing the pre-installed Python 2 binary from macOS 12.3, I was forced to pull the code out, dust it off and update it to Python 3. I chose to use the MacAdmins “Managed Python” since it provides a nice bundle of libraries which should be useful for some other tools I have (and still need to migrate). It should (probably) work with other Python 3 distributions, as well, but I did not test this at all.

Thankfully, quickpkg did not require very many code changes. I could even remove all the code from Greg Neagle’s FoundationPlist. Python 2 plistlib could not read binary plist files, so that extra functionality was required. But Python 3 plistlib can read and write binary plists and I could save a lot of code.

The script has worked with all of my tests, but I probably have not explored all the weird edge cases yet. So I am releasing it as a beta. Please file issues and/or pull requests on the GitHub repo if you run into any problems.

Update: pkgcheck

The macOS Monterey 12.3 beta release notes say that the Python 2.7 binary (located in /usr/bin/python) will be removed. Since you follow this blog, this should not come as a surprise. We have been warned about this since Catalina. (Or longer)

That said, the removal of Python 2 in a minor macOS release is surprising. Minor updates should not have breaking changes or removals. Admins and developers may not expect removals and other breaking changes in a minor update and therefore not be paying as much attention to changes. Also, the time a minor update is in beta is usually 6-8 weeks, which leaves us and developers much less time to find and fix problems than a major update beta phase, which is usually 4-5 months.

Nevertheless, we have to work with what Apple deals to us. MacAdmins have been investigating their own tools and scripts since the Monterey release or earlier to avoid the prompts. But when you get vendor pkgs, these might contain anything. While you can inspect pkgs with tools like pkgutil, Pacifist or Suspicious Package, it can get tedious with many packages.

A while back I built a script called pkgcheck to automate this check. Since I (and many others) have started using it again in the recent days, I have added a few more checks to it.

The earlier version would flag files in the installer’s resources that had a /bin/bash, /usr/bin/python, /usr/bin/ruby, or /usr/bin/perl shebang. (the first line with the #!) I have now also added check for a shebang with /usr/bin/env [python|ruby|perl] because when run from an installer pkg, this will also resolve tousing the built-in, deprecated runtimes. Also, using python in the shebang will now be shown as a red error, rather than a yellow warning.

The script will now also grep for use of python in installation scripts and show those scripts. This might generate a few false positives. You will have to use your judgement. For example using python3 in an installation script will also trigger this. But then, it probably should, since python3 is not installed on macOS by default. (What you see in /usr/bin/python3 is a shim that prompts you to install the Command Line Developer Tools, unless they or Xcode are already installed.)

I hope this is useful!

Suspicious Package 4.0 Update

I recently posted about some Suspicious Package Power User Features, which was a follow-up to my MacDevOps YVR presentation “The Encyclopedia of Packages.”

As a follow-up to that follow-up, Suspicious Package was updated to version 4.0 today. (Yesterday? Time zones are strange.) The update to brings compatibility with macOS Monterey and some really nice refinements to these power user features.

First and foremost, Suspicious Package will now show the kind of package, or “package format” in the Package Info tab. This makes me very happy, not just because the FAQ references my presentation. As the FAQ correctly states, most users of packages or even the Suspicious Package application will not care much about the differences between the package formats, but for system administrators, this can determine the difference between a functional deployment or a broken workflow.

The previously ‘secret’ option to show the PackageInfo xml file is now also exposed in the preferences window, next to the option to show the Distribution XML.

It is now also easier to search for the contents of a particular component in a distribution package.

You can download the latest version of Suspicious Package and get the update notes here.

Many thanks to Randy Saldinger of Mothers Ruin Software for providing this amazing tool and further refining it!

Randy was also recently a guest on the MacAdmins Podcast. You check it out if you have not yet listened to that episode.

Suspicious Package Power User Features

As many MacAdmins, I work a lot with installer packages. You can say I wrote the book about it. When you get an installer package from some vendor website, you will want to inspect it before you install it anywhere, let alone deploy it to dozens, hundreds, or even thousands of Macs in your fleet.

You can use the pkgutil tool to do this in the command line, but there are package inspector tools with a graphical interface that are very useful and popular.

One of these tools is Suspicious Package from Mothers Ruin Software. It displays all of the content and resources of a pkg file in a very nice user interface. Many people love Suspicious Package from Mothers Ruin Software. I have always had reservations about Suspicious Package, though, because I thought there were a few missing features.

Update (August 2021): The app has gotten a new major update (4.0) which makes these “power user features” even more accessible!

The missing features were in connection with distinguishing “normal” or “component” packages (which have a single payload) from “distribution” or “meta” packages (which don’t have a payload of their own, but contain one or more component packages).

I have explained the differences in a bit more detail in my MacDevOps YVR presentation “The Encyclopedia of Packages” where I (once again, ignorantly) stated that you can’t really tell them apart in the UI of Suspicious Package.

After that presentation, Mat X, one of the organizers of the conference got me in touch with the developer of Suspicious Package, Randy Saldinger, who graciously and patiently demonstrated that I was wrong.

In my defense, you really cannot tell normal packages from distribution packages in the default configuration of Suspicious Package, but if I had bothered to read the manual and/or explore the Preferences window, I would have found this option:

This will show the Distribution xml file at the top of the list of the ‘All Scripts’ pane for distribution packages. When you see no Distribution file there, the package is a component package.

The second checkmark in that preference window is also very useful. With “Component package and bundle info” enabled you can see which component contains the selected file in the info pane:

You can also search in the “All Files” tab with command-F and use the component package ID as a search criteria.

All of this is already well enough to remove the reservations I have had on Suspicious Package. But Randy shared another preference with me which puts it over the top. It is not exposed in the UI (yet) but when you run:

% defaults write com.mothersruin.SuspiciousPackage ShowRawPackageInfo -bool YES

in Terminal and re-launch Suspicious Package, you will see the raw PackageInfo xml in the “All Scripts” tab. Together with showing the Distribution xml, this allows you to inspect all the raw metadata that can be inside a pkg file.

I have also learned that you can use the search functionality in the “Help” menu of Suspicious Package which will link directly to the online documentation. Not many apps leverage this functionality, so we often forget to check for it. Kudos to Suspicious Package for using this.

Many thanks to Randy for all his work and help and for providing an excellent tool! I am very much looking forward to the next version.

Platform Support in macOS Installer Packages (pkg)

Mac users and admins find themselves in yet another major platform transistion. For the duration of the transition, developers and admins will have to deal with and support software and hardware for the Intel and Apple silicon Macs. With Universal applications and Rosetta 2, Apple is providing very efficient tools to dramatically reduce the friction and problems involved.

This post was inspired by comments from Josh Wisenbaker on MacAdmins Slack and Twitter. Thank you!

For most end user level tasks, these tools will provide seamless experience. Universal applications will run on either platform natively and Rosetta 2 will translate applications compiled for the legacy platform (Intel) so they can run on the new Apple silicon chips. There are only a few situations where these tools don’t work: virtualization solutions and Kernel extensions.

In most cases this tools will “just work.” But for MacAdmins there is one major issue that may throw a wrench in your well-oiled deployment workflows. Rosetta is not pre-installed on a fresh macOS installation.

We can only speculate why Apple chooses to deliver Rosetta this way. In “normal” unmanaged installations, this is not a big deal. The first time a user installs or launches a solution that requires Rosetta, they will be prompted to for installation and upon approval, the system will download and install Rosetta.

As a MacAdmin, however, you want your deployments to be uninterrupted by such dialogs. Not only are they confusing to end users, but the user might cancel out of them which will result in your workflow failing partially.

There are two solutions. The first is to install Rosetta as early as possible in the deployment process. Apple provides a new option for the softwareupdate command to initiate the installation. Graham Gilbert and Rich Trouton have already published scripts around this. Have this script run early in your deployment workflow on Apple silicon and subsequent apps and tools that require Rosetta should be fine.

The other solution is to avoid requiring Rosetta and thus the prompt for Rosetta.

I mentioned earlier that we can only speculate as to why Apple has made Rosetta 2 an optional installation. One possible explanation is, that Apple believes Rosetta will not be a necessary installation for very long. An extra dialog and installation will make users and developers more aware of software that “needs an update” and motivate developers to provide Universal applications faster.

When a user opens an application that requires Rosetta for the first time, before Rosetta is installed, the system prompts to install. The same thing can happen with an installer package. The system might prompt to install Rosetta before a certain package is installed. However, not all packages trigger the dialog. I was curious what is required in the package to trigger or to avoid the prompt.

Aside from legacy formats, there are two types of packages. The first are “plain” packages, which are also called component packages. These packages have a payload and can have pre- and postinstall scripts, but other than that, there is little metadata you can add to influence the installation workflow.

This is where “distribution packages” come in. Distribution packages do not have a payload or installation scripts of their own, but contain one or more component packages. In addition, distribution packages can contain metadata that influences the installation workflow, such as customization of the Installer.app interface, system version checks, prompting the user to quit running applications before an installation and software requirements and a few more.

Note: learn more about the detailed differences between component and distribution packages in my book: “Packaging for Apple Administrators“

You can build a distribution package from a component package with the productbuild command:

> productbuild --package component.pkg distribution.pkg

Since most of the extra features of distribution packages are only effective when the installation package is launched manually in the Installer application, MacAdmins usually just build component pkgs.

The confusing part here is that both component pkgs and distribution pkgs have the same file extension. They are hard to distinguish even from the command line. To tell them apart, you can expand a pkg with the pkgutil command and look at the files in the expanded folder. Component pkgs have (among other files) a PackageInfo file and distribution pkgs have a Distribution file:

# component pkg
> pkgutil --expand component.pkg expanded_component_pkg
> ls expanded_component_pkg
Bom
Payload
Scripts
PackageInfo

# distribution pkg
> pkgutil --expand distribution.pkg expanded_distribution_pkg
> ls expanded_distribution_pkg
component.pkg
Distribution

For distribution pkgs, the Distribution file is an XML file which contains the configuration data for the package. One tag in this XML is the options tag which can have a hostArchitectures attribute. According to [Apple’s documentation on this tag](A comma-separated list of supported architecture codes), the hostArchitectures are a “comma-separated list of supported architecture codes.”

Apple documentation is a bit aged, so it gives i386, x86_64, and ppc as possible values. However, when you read the productbuild man page on macOS Big Sur you will see that arm64 is a new valid value. We will also find these extremely helpful note:

NOTE: On Apple Silicon, the macOS Installer will evaluate the product’s distribution under Rosetta 2 unless the arch key includes the arm64 architecture specifier. Some distribution properties may be evaluated differently between Rosetta 2 and native execution, such as the predicate specified by the sysctl-requirements key. If the distribution is evaluated under Rosetta 2, any package scripts inside of product will be executed with Rosetta 2 at install time.

When a distribution pkg has this attribute and it contains a value of arm64 then the installation process on an Apple silicon Mac will not check if Rosetta is installed. When arm64 is missing from the hostArchitectures, or the attribute or tag are missing entirely, the installation process on an Apple silicon Mac will asume the pkg requires Rosetta and prompt to install when necessary.

There is more good news in the next note in the man page:

NOTE: Starting on macOS 11.0 (Big Sur), productbuild will automatically specify support for both arm64 and x86_64 unless a custom value for arch is provided.

When you use productbuild to create a distribution pkg on Big Sur (Intel and Apple silicon) both arm64 and x86_64 will be added to the configuration by default.

But, when you use productbuild on Catalina or earlier, the attribute will be lacking, when means that when someone installs that pkg on an Apple silicon Mac, it will assume it requires Rosetta and prompt for installation.

Adding both architectures by default is a useful default. But can we set the value explicitly when we build the distribution pkg? And can we do so on Catalina?

Yes, you can, of course. There are even two solutions. First, instead of letting productbuild generate the Distribution xml, you can build and provide a complete Distribution xml file with the --distribution option. That will give you full, fine-grained control over all the options.

The second solution is a bit easier. You can create a requirements.plist property plist file in the form:

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
        <key>arch</key>
        <array>
                <string>x86_64</string>
                <string>arm64</string>
        </array>
</dict>
</plist>

Then you can provide this property list file to the productbuild command with the --product option.

> productbuild --package component.pkg --product requirements.plist distribution.pkg

This way, productbuild still generates the Distribution xml and merges in your choices from the requirements.plst. There are other options you can add which are documented in the productbuild man page.

Both of these approaches will work on Catalina as well. This way you can explicitly tell the installer system which architectures your packages will run with and not leave anything to chance.

In Whitebox Packages you can configure the hostArchitectures attribute under the “advanced options” for a distribution package.

As far as I can tell, when you install a component pkg, no checks for Rosetta are performed. Nevertheless, this is not something I would rely on. For packages that are crucial to the deployment workflow, I would recommend going the extra step and creating a distribution pkg from the component pkg with the proper flags set. This way you can ensure proper behavior.

Of course, if your package installer contains any form of Intel-only, not-universal binary, you should not abuse this just to skip the annoying Rosetta dialog, as it might lead to problems later. But, when the software you are installing is universal, you sould use this to tell the system which platforms your package supports.

Strategies to using desktoppr

A while back I introduced desktoppr. It is a very simple tool; its singular goal is to set the desktop picture (users and admins migrating from Windows call it ‘wallpaper,’ but that is not the proper macOS nomenclature) without requiring PPPC/TCC whitelisting.

The good news is that desktoppr still works fine, nearly one-and-a-half years in! Even though Catalina brought in many changes and restrictions, desktoppr kept doing its job.

Nevertheless, as I have used desktoppr myself in several different deployments, and I have a few different approaches to deployment, depending on the requirements.

Catalina split system volume

One of the new features of Catalina is a read-only system volume, separate from the data volume. This means that the pre-installed desktop pictures are no longer in /Library/Desktop Pictures/ but can now be found in /System/Library/Desktop Pictures. This is on the read-only part of the file system.

On a new “fresh” macOS installation, the /Library/Desktop Pictures does not exist. However, when you create this folder, its contents will appear in the ‘Desktop’ preference pane, merged with the contents of the protected system folder. So, we can continue to use /Library/Desktop Pictures as a place to store and install custom desktop image files.

Note: if you do not want the custom desktop picture to appear in the Desktop preference pane, then you can install the file in a different location. /Users/Shared or /Library/MyOrganization/ are useful locations.

Packaging the custom picture

> mkdir -p BoringDesktop/payload

> cd BoringDesktop
> cp /path/to/BoringBlueDesktop.png payload

> pkgbuild --root payload --install-location "/Library/Desktop Pictures/" --identifier com.example.BoringDesktop --version 1 BoringDesktop.pkg


These command will create a payload folder, copy an image file (my example is BoringBlueDesktop.png) and build an installation pkg using the pkgbuild command.

If you want a more detailed explanation of this process, you can find it in my book: “Packaging for Apple Administrators”

Lock the Desktop

In classroom, lab, and kiosk settings, MacAdmins may want to set and lock the desktop picture. In this use case, you do not need desktoppr at all.

Use the above pkg to install the image file and then use your management system to push a configuration profile that sets and locks a Desktop Picture.

Many management systems will have the desktop picture controls hidden in the ‘Restrictions’ payload among many other settings. Please consult the documentation. You can also use this custom profile that only controls the desktop setting.

Preset the desktop, but the let user change it

This is the most common way MacAdmins will want to deploy is to pre-set a custom Desktop Picture but allow the user to change it later. This is what desktoppr was created for.

There are two approaches you can take to do this. Well, to be honest, there are way more, and all of them are valid, as long as they work. I should say: I will show two different approaches.

The modular approach

In this case you use your management system to install and run all the pieces:

  • install the custom desktop picture using the above pkg
  • install desktoppr using the pkg*
  • run a script that sets the desktop

* for 10.14.3 and earlier, desktoppr v0.2 requires the Swift 5 runtime support for command line tools to be installed.

The advantage of this approach is that we already did the first part earlier, and the desktoppr pkg can be downloaded from the git repo,. So, we already have two of the three parts.

For the script, there is a sample script in the repository, as well.

Note that this script has changed slightly since the last post. Originally, the script used launchctl asuser. The behavior of launchctl asuser seems to have changed somewhat in a recent update and I have switched the script to use sudo -u instead.

This approach can be used with Munki, Jamf, outset, and many other management solutions.

All in one package

The downside of the modular approach is that you have to manage three pieces (the image file, the desktoppr binary, and a script) in your management system. This can be especially problematic when you are not the actual administrator of the management system but more active in a ‘consulting role.’

In this case, I have found it easier to build a single package that does all the work. This is easier to hand over to another admin. It is also more flexible and can be used in even more situations. It is a bit more work to assemble, though.

First, you need the ‘ingredients:’

* for 10.14.3 and earlier, desktoppr v0.2 requires the Swift 5 runtime support for command line tools to be installed.

First, create a project folder with a payload folder inside. Then copy the necessary files into the right place:

> mkdir -p BoringDesktop/payload
> cd BoringDesktop


Copy the image file to payload:

> cp /path/to/BoringBlueDesktop.png payload


Create a scripts directory and copy the postinstall script to it:

> mkdir scripts
> cp path/to/desktoppr/examples/postinstall scripts


Expand the zip archive (in Finder) and copy the desktoppr binary into the scripts folder.

> cp path/to/build/usr/local/bin/desktoppr scripts


Your project folder should now look like this:

BoringDesktop Project FolderBoringDesktop Project Folder

You can then build the pkg with:

> pkgbuild --root payload --scripts scripts --install-location "/Library/Desktop Pictures/" --identifier com.example.BoringDesktop --version 2 BoringDesktop-2.pkg


Note the different version number, so the system can recognize this as a different pkg from the one you might have built earlier.

This form of the pkg does not actually install the desktoppr binary on the target system. When the pkg is created, the entire contents of the scripts folder will be archived into the pkg file. When the pkg is being installed, the contents will be expanded into a temporary directory. That means that the postinstall script can see the binary in the same director ‘next to it.’ This happens in lines 22–27 of the postinstall script.

After the installation is complete, the temporary files will be removed, so the postinstall script and the desktoppr binary will be removed automatically. You don’t need to worry about the cleanup.

Conclusion

Which approach works best depends on your specific deployment needs, your management setup and workflows and (not the least) your comfort with building scripts and packages.

Even when you have defined your deployment needs, there are multiple solutions on how to build and deploy a custom desktop picture. As long as they achieve the desired goal, there is no “best” solution.

You can earn more details about building packages in my book: “Packaging for Apple Administrators”

Notarization for MacAdmins

Apple introduced Notarization in macOS Mojave. Since its introduction Apple has kept increasing the use of notarization checks in macOS. For macOS Catalina, Apple has been very vocal saying that Notarization is a requirement for distribution of Applications outside of the Mac App Store.

This has left many MacAdmins confused and concerned. A large part of the work as a MacAdmin consists of (re-)packaging applications, configuration files and scripts so they can be distributed in an automated fashion through a management system, such as Jamf Pro, Munki, Fleetsmith, etc.

Do MacAdmins need to notarize all the package installers they create as well? Do MacAdmin need to obtain an Apple Developer ID? How should MacAdmins deal with notarized and non-notarized applications and installers from third parties?

This post is an attempt to clarify these topics. It’s complicated and long, bear with me…

Signed Applications

Apple’s operating systems use cryptographic signatures to verify the integrity and source of applications, plug-ins, extensions, and other binaries.

When an application, plug-in, extension, or other binary (from now on: “software”) is signed with a valid Apple Developer certificate, macOS (or iOS, tvOS, and watchOS) can verify that the software has not been changed or otherwise tampered with since it was signed. The signature can verify the source of the signature, i.e the individual Developer account or Developer team whose Developer identity was used to sign the software.

If the contents of the software were changed for some reason, the verification fails. The software can be change by accident or with malicious intent, for example to inject malicious code into an otherwise beneficial piece of software.

Since Apple issues the Developer IDs, they also have the option of revoking and blacklisting certificates. This usually happens when a Developer ID has been abused to distribute malware. The Malware Removal Tool or MRT is the part of the system that will identify and block or remove blacklisted software.

App Store Distribution

Applications distributed through Apple’s App Stores have to be signed with a valid Developer ID. A developer needs to have valid subscription ($99 per year for individuals and $299 for organizations) to obtain a certificate from Apple.

When a developer submits software to an App Store on any Apple system, the software will be reviewed by Apple to confirm whether it meets the various guidelines and rules. This includes a scan for malware.

App Store applications also have to be sandboxed, which means they can only access their own data (inside the “sandbox”) and not affect other applications, services, or files without certain “entitlements” and, in many cases, user approval.

App Store rules and regulations and sandbox limitations preclude many types of applications and utilities. On iOS, tvOS and watchOS, they are the only way for developers to distribute software to end users.

Apple provides a method for Enterprises and Organizations to distribute internal software directly without going through the App Store and App Store review. This should be limited to distribution to employees and members of the organization (such as students of a university or school). This method has infamously been abused by Facebook and other major companies which lead to Apple temporarily revoking their certificates. (We will not discuss Enterprise App Distribution in this post.)

There is also much criticism about how realistic Apple’s rules and guidelines are, how arbitrary the review process is, and whether the sandbox restrictions are useful or unnecessarily draconic. A lot of this criticism is valid, but I will ignore this topic in this post for the sake of simplicity and brevity.

Software downloaded from the App Store is automatically trusted by the system, since it underwent the review and its integrity and source can be verified using the signature. In the rare case that some malicious software was missed but the review process, Apple can revoke the Developer certificate or blacklist the software with the Malware Removal Tool.

Distribution outside of the Mac App Store: Gatekeeper and Quarantine

As mentioned before, iOS, tvOS, and watchOS applications have to distributed to end users through the App Store, be signed with a valid Developer ID and under go the review.

Because the Mac existed a long time before the Mac App Store, software vendors have many ways of distributing software. Originally software was sold and delivered on physical media (Floppy Disks, CDs, and DVDs), but we with the rise of the internet, users could simply download software from the developer’s or vendor’s website or other, sometimes dubious, sources.

Apple has (so far) accepted and acknowledged that these alternative means of software distribution and installation are necessary on macOS. To provide an additional layer of security for the end user in this use case, Apple introduced Gatekeeper in OS X 10.8 Mountain Lion.

When a user downloads a software installer or archive from the internet it is ‘quarantined.’ When the user attempts to install or launch the software for the first time, Gatekeeper will evaluate the software. There are many steps in this evaluation, and Howard Oakley explains the process in much detail in this post.

You can see the quarantine flag with the xattr command:

% xattr ~/Downloads/somefile.pkg
com.apple.macl
com.apple.metadata:kMDItemWhereFrom
com.apple.quarantine

You can delete the quarantine flag with xattr -d com.apple.quarantine path/to/file. Usually, there is no real need to.

The first step of the evaluation is verifying the software’s signature, Developer ID, integrity. When encountering an unsigned piece of software the user will be presented with a warning dialog.

Users with administrator privileges can bypass Gatekeeper by choosing “Open” from the context menu instead of double-clicking to open. Gatekeeper can be completely disabled with the spctl command, though this is not recommended.

The Developer signature provides a way to verify the source and integrity of a piece of software, but since the distribution happens outside of Apple’s control, a malicious developer could still put any form of malicious code in the signed software to keep Gatekeeper happy. As long as the malware avoids widespread detection it will look good to Gatekeeper and the end user. Even when the malware is detected by Apple and the Developer ID is revoked, it is not hard for a malicious developer to obtain or steal a new Developer ID and start over.

Enter Notarization

Apple needed another layer of security which could scan software for known malware and enforce a certain set of security rules on third party software, even when it is distributed outside of the Mac App Store.

Note: I find the effort Apple is putting in to Gatekeeper and Notarization quite encouraging. If Apple wanted to restrict macOS to “App Store only” distribution in the near future, this effort would not be necessary. This shows that Apple still acknowledges the important role that independent software distribution has for macOS.

To notarize software, a developer has to sign it with their Developer ID, and upload it to Apple using Xcode or the altool command. Then Apple notarization workflow will verify that the software fulfills certain code requirements and scans for certain malware. The exact details of what is considered malware are unknown. However, we do know that the process is fully automated and, unlike the App Store approval process, does not involve human reviewers.

If the software has passed the notarization process the result will be stored on Apple’s servers. When Gatekeeper on any Mac verifies the software it can confirm the notarization status from Apple’s servers. Alternatively, a developer can ‘staple’ a ‘ticket’ to the software, which allows Gatekeeper to confirm the notarization status without needing to connect to Apple.

When Gatekeeper encounters a quarantined software that is notarized, it will show the familiar Gatekeeper dialog with an additional note that:

“Apple checked [the software] for malicious software and none was detected.”

Since 10.14.5, When Gatekeeper encounters signed software that is not notarized it will show the standard dialog but with an additional yellow warning sign.

As with the previous Gatekeeper checks for a valid signature an administrator user can override the check by choosing ‘Open’ from the context menu instead of double-clicking to open.

In Mojave notarization was enforced in Gatekeeper checks for kernel extensions and in 10.14.5 for software with new Developer IDs, which where created after June 2019.

Starting with Catalina, all software needs to be notarized to pass Gatekeeper when the first launch or installation is initiated by a user.

However, the warning can still be overridden by an administrator user using the context menu.

What can be Notarized

As of now, the following pieces of software can be notarized:

  • Application bundles
  • Kernel Extensions
  • Installer Packages (pkg), Disk images (dmg) and zip archives

When you are building other types of software, such as command line tools, you can (and should) place them in one of the archive formats. The preferred choice for MacAdmins should be an installer package (pkg) since it will also place the binary in the correct location in the file system with the correct access privileges.

What cannot be Notarized

You should not notarize a binary or application that you did not sign! The Developer ID used to sign a binary (application or command line tool) should be the same as the Developer ID used to submit the software for notarization.

Apple has loosened the requirements for notarization until Jan 2020 to give developers some extra time to adapt. Once the requirements return to the full restrictions an attempt to notarize third party software with a different Developer ID will fail. (Existing notarizations will remain valid after that date.)

Installer command

When you install software using the installer command from the Terminal or a script, it will bypass quarantine and the Gatekeeper check.

This is also true when you install software using a management system such as Jamf Pro, Munki, Fleetsmith, etc.

Software you re-package as a MacAdmin for distribution through management systems does not need to be notarized.

Given this and the limitations on notarizing third party software above, you should very rarely need to notarize as a MacAdmin.

Example: Re-packaging third party software from dmg

A lot of applications for macOS are distributed as disk images. The normal end user workflow would be to mount the dmg after downloading, and then copying the application from the dmg to the /Applications folder.

There are two steps where Gatekeeper might trigger: when you mount the disk image and when you launch the application after copying for the first time. To pass both these checks, a developer should prudently notarize both the disk image and the application. Google Chrome for example does exactly that, avoiding the Gatekeeper warning.

We can verify this with the spctl command:

% spctl -a -vv -t install ~/Downloads/googlechrome.dmg
/Users/armin/Downloads/googlechrome.dmg: accepted
source=Notarized Developer ID
origin=Developer ID Application: Google, Inc. (EQHXZ8M8AV)

% spctl -a -vv -t execute "/Volumes/Google Chrome/Google Chrome.app"
/Volumes/Google Chrome/Google Chrome.app: accepted
source=Notarized Developer ID
origin=Developer ID Application: Google, Inc. (EQHXZ8M8AV)

Unfortunately, some management systems don’t understand “Apps in disk images” as a distribution method. For these systems MacAdmins need to re-package the application into a pkg. You can do that quickly with pkgbuild:

% pkgbuild --component /Volumes/Google\ Chrome/Google\ Chrome.app --install-location /Applications/ GoogleChrome.pkg
pkgbuild: Adding component at /Volumes/Google Chrome/Google Chrome.app
pkgbuild: Wrote package to GoogleChrome.pkg

or use quickpkg.

This new installer package will be neither signed nor notarized:

% spctl -a -vv -t install GoogleChrome.pkg                          
GoogleChrome.pkg: rejected
source=no usable signature

When you send this installer package to another Mac with AirDrop, the receiving system will attach the quarantine flag. And when you double click it, you will get the Gatekeeper warning. However, when you can still install it using the installer command in Terminal, which bypasses the Gatekeeper system, just as your management system will:

% installer -pkg ~/Downloads/GoogleChrome.pkg -tgt /

Alternatively, you can choose “Open” from the context menu in Finder to bypass Gatekeeper. However, this is not something you want to teach your end users to do regularly.

Firefox can be downloaded as a disk image as well as a installer package. While the application inside both is notarized, neither the disk image nor the installer package are. The disk image mounts with no issues, but when you try to open the installer pkg by double-clicking you will get the expected notarization warning. Nevertheless, the pkg will work fine after importing to your management system.

Edge cases

There are some cases where notarization would be useful for MacAdmins but might not even be possible. I met a MacAdmin working at a university at MacSysAdmin last week. They need to re-package a VPN client with customized configuration files to be installed on student-owned machines.

There is really no solution without the students running into the notarization warning. Teaching the users how to bypass Gatekeeper is not a good solution.

In these cases you have to work with the software vendor and Apple to find a workable solution.

Summary

Notarization is a new security layer introduced by Apple in Mojave. The restrictions imposed on non-notarized software increase in Catalina.

When an Application is installed or launched for the first time by the user (by double-clicking) Gatekeeper will verify the signature and notarization status and warn the user if any are missing.

Developers should sign and notarize their applications and tools.

Mac Administrators should not notarize applications and tools from third parties.

Applications and packages installed through management systems bypass Gatekeeper and do not need to be notarized.

Conclusion

Apple is loudly messaging that notarization is absolutely required for applications in Catalina. While this message makes sense for the developers building the software, it does not apply to administrator who re-package third party software for distribution through management systems.

MacAdmins should join Apple in demanding signed and notarized binaries and installer packages from developers.

However, MacAdmins can also continue their current workflows for re-packaging and distribution.

Links

Check Installer Pkgs for deprecated scripts

macOS 10.15 Catalina will deprecate the built-in /bin/bash. I have talked about this at length.

The release notes for Catalina also tell us that other built-in scripting runtimes, namely Python, Perl, and Ruby. Will not be included in future macOS releases (post-Catalina) any more.

This means, that if you want to use bash, Python, Perl, or Ruby on macOS, you will have to install, and maintain your own version in the future.

However, scripts in installation packages, cannot rely on any of these interpreters being available in future, post-Catalina versions of macOS. Installer pkgs can be run in all kinds of environments and at all times, and you would not want them to fail, because a dependency is missing.

The good news is that we still have time. All the runtimes mentioned above are still present in Catalina, so the packages will continue to work for now. But if you are building installation scripts, you need to check if any of the installation scripts use one of these interpreters and fix them.

I recommend to use /bin/sh for installation scripts, since that will run in any macOS context, even the Recovery system.

If you are using third-party installer packages, you may also want to check them for these interpreters, and notify the developer that these packages will break in future versions of macOS.

To check a flat installer package, you would expand it with pkgutil --expand and then look at script files in the Scripts folder. This will work fine for a package or two, but gets tedious really quickly, especially with large distribution pkgs with many components (e.g. Office).

So… I wrote a script to do it. The script should handle normal component pkgs, distribution pkgs and the legacy bundle pkgs and mpkgs.

You can get the pkgcheck script from my Github repo.

What the script does

Once I had written the code to inspect all these types of pkgs, I realized I could grab all other kinds of information, as well. The pkgcheck.sh script will check for:

  • Signature and Notarization
  • Type of Package: Component, Distribution, legacy bundle or mpkg
  • Identifier and version (when present)
  • Install-location
  • for Distribution and mpkg types, shows the information for all components as well
  • for every script in a pkg or component, checks the first line of the script for shebangs of the deprecated interpreters (/bin/bash, /usr/bin/python, /usr/bin/perl, and /usr/bin/ruby) and print a warning when found

How to run pkgcheck.sh

Run the script with the target pkg file as an argument:

% ./pkgcheck.sh sample.pkg

You can give more than one file:

% ./pkgcheck.sh file1.pkg file2.pkg ...

When you pass a directory, pkgcheck.sh will recursively search for all files or bundle directories with the pkg or mpkg extension in that directory:

% ./pkgcheck.sh SamplePkgs

Features and Errors

There are a few more things that I think might be useful to check in this script. Most of all, I want to add an indicator whether a component is enabled by default or not. If you can think of any other valuable data to display, let me know. (Issue or Pull Request or just ping me on MacAdmins Slack)

I have tested the script against many pkgs that I came across. However, there are likely edge cases that I haven’t anticipated, which might break the script. If you run into any of those, let me know. (File an Issue or Pull Request.) Having the troublesome pkg would of course be a great help.

Note: the script will create a scratch directory for temporary file extractions. The script doesn’t actually expand the entire pkg file, only the Scripts sub-archive. The scratch folder will be cleaned out at the beginning of the next run, but not when the script ends, as you might want to do some further inspections.

Sample outputs

This is a sample pkg I build in my book, it has pre- and postinstall scripts using a /bin/bash shebang:

% ./pkgcheck.sh SourceCodePro-2.030d.pkg
SourceCodePro-2.030d
SamplePkgs/SourceCodePro-2.030d.pkg
Signature:      None
Notarized:      No
Type:           Flat Component PKG
Identifier:     com.example.SourceCodePro
Version:        2.030d
Location:       /
Contains 2 resource files
postinstall has shebang #!/bin/bash
preinstall has shebang #!/bin/bash

This is the experimental notarized pkg installer for desktoppr:

% ./pkgcheck.sh desktoppr-0.2.pkg
desktoppr-0.2
SamplePkgs/desktoppr-0.2.pkg
Signature:      Developer ID Installer: Armin Briegel (JME5BW3F3R)
Notarized:      Yes
Type:           Flat Component PKG
Identifier:     com.scriptingosx.desktoppr
Version:        0.2
Contains 0 resource files

And finally, this is a big one, the Microsoft Office installer: (they have some work to do to clean up those scripts)

% ./pkgcheck.sh Microsoft\ Office\ 16.27.19071500_Installer.pkg
Microsoft Office 16.27.19071500_Installer
SamplePkgs/Microsoft Office 16.27.19071500_Installer.pkg
Signature:      Developer ID Installer: Microsoft Corporation (UBF8T346G9)
Notarized:      No
Type:           Flat Distribution PKG
Contains 11 component pkgs

    Microsoft_Word_Internal
    Type:           Flat Component PKG
    Identifier:     com.microsoft.package.Microsoft_Word.app
    Version:        16.27.19071500
    Location:       /Applications
    Contains 3 resource files

    Microsoft_Excel_Internal
    Type:           Flat Component PKG
    Identifier:     com.microsoft.package.Microsoft_Excel.app
    Version:        16.27.19071500
    Location:       /Applications
    Contains 2 resource files

    Microsoft_PowerPoint_Internal
    Type:           Flat Component PKG
    Identifier:     com.microsoft.package.Microsoft_PowerPoint.app
    Version:        16.27.19071500
    Location:       /Applications
    Contains 2 resource files

    Microsoft_OneNote_Internal
    Type:           Flat Component PKG
    Identifier:     com.microsoft.package.Microsoft_OneNote.app
    Version:        16.27.19071500
    Location:       /Applications
    Contains 2 resource files

    Microsoft_Outlook_Internal
    Type:           Flat Component PKG
    Identifier:     com.microsoft.package.Microsoft_Outlook.app
    Version:        16.27.19071500
    Location:       /Applications
    Contains 2 resource files

    OneDrive
    Type:           Flat Component PKG
    Identifier:     com.microsoft.OneDrive
    Version:        19.70.410
    Location:       /Applications
    Contains 30 resource files
    postinstall has shebang #!/bin/bash
    od_logging has shebang #!/bin/bash
    od_service has shebang #!/bin/bash
    od_migration has shebang #!/bin/bash
    preinstall has shebang #!/bin/bash

    Office16_all_autoupdate
    Type:           Flat Component PKG
    Identifier:     com.microsoft.package.Microsoft_AutoUpdate.app
    Version:        4.13.19071500
    Location:       /Library/Application Support/Microsoft/MAU2.0
    Contains 2 resource files
    postinstall has shebang #!/bin/bash
    preinstall has shebang #!/bin/bash

    Office16_all_licensing
    Type:           Flat Component PKG
    Identifier:     com.microsoft.pkg.licensing
    Version:        16.27.19071500
    Location:       /
    Contains 2 resource files
    dockutil has shebang #!/usr/bin/python

    Office_fonts
    Type:           Flat Component PKG
    Identifier:     com.microsoft.package.DFonts
    Version:        0
    Location:       /private/tmp/com.microsoft.package.DFonts
    Contains 1 resource files
    postinstall has shebang #!/bin/bash

    Office_frameworks
    Type:           Flat Component PKG
    Identifier:     com.microsoft.package.Frameworks
    Version:        0
    Location:       /private/tmp/com.microsoft.package.Frameworks
    Contains 1 resource files
    postinstall has shebang #!/bin/bash

    Office_proofing
    Type:           Flat Component PKG
    Identifier:     com.microsoft.package.Proofing_Tools
    Version:        0
    Location:       /private/tmp/com.microsoft.package.Proofing_Tools
    Contains 1 resource files
    postinstall has shebang #!/bin/bash

Install Bash 5 on macOS with Patches

I recently posted an article on how to download, install, and build a macOS installer pkg for bash 5. In that first version of this post I ignored patches, minor updates to the bash source code and binary. But as the patches to bash 5 are accumulating, I cannot ignore them much longer.

This post will extend the instructions in the original post.

After downloading and expanding the bash-5.0.tar.gz, create a patches folder:

$ cd path/to/bash-5.0
$ mkdir patches
$ cd patches

You can download the patches for bash-5.0 here. As of this writing, there are seven patches for bash-5.0 labelled bash50-001 through bash50-007. You can download all at once with:

$ curl 'https://ftp.gnu.org/gnu/bash/bash-5.0-patches/bash50-[001-007]' -O

(Adapt the numbers when there are more patches in the future.)

Then move up one directory level to the bash-5.0 root directory and apply the patches using the patch command.

$ cd ..
$ patch -p0 -i patches/bash50-001
$ patch -p0 -i patches/bash50-002

(etc.)

You can download and patch with a single step. Make sure your working directory is the bash-5.0 with all the code and run:

$ curl 'https://ftp.gnu.org/gnu/bash/bash-5.0-patches/bash50-[001-007]' | patch -p0

From here, you can continue with the remaining build steps from the original post. The next step will be running ./configure.

The script to build the pkg installer has also been updated in the repository to download and apply the patches before building.

Build Simple Packages with Scripts

In a past post, I described how path_helper works. As an example, I mentioned the installer for Python 3 which runs a postinstall script that locates and modifies the current user’s shell profile file to add the Python 3 binary directory to the PATH.

Not only is modifying a user’s file a horrible practice, but it will not achieve the desired purpose when the user installing the package is ultimately not the user using the system. This setup happens fairly frequently in managed deployment workflows.

As described in the earlier post, macOS will add the contents of files in /etc/paths.d/ to all users’ PATHs. So, all we have to do is create a file with the path to the Python 3 binary directory in /etc/paths.d/. A perfect task for a simple installer package.

The steps to create such an installer are simple:

$ mkdir -p Python3PathPkg/payload
$ cd Python3PathPkg
$ echo "/Library/Frameworks/Python.framework/Versions/3.7/bin" > payload/Python-3.7
$ pkgbuild --root payload --install-location /private/etc/paths.d --version 3.7  --identifier com.example.Python3.path Python3Path-3.7.pkg
pkgbuild: Inferring bundle components from contents of payload
pkgbuild: Wrote package to Python3Path-3.7.pkg

This is not so hard. However, since the path to binary contains the major and minor version number, you will have to create a new version when Python 3 updates to 3.8 (and 3.9, etc…).

So, it makes sense to script the process. With a package this simple, you can create everything required to build the package (i.e. the payload folder with contents) from the script in a temporary directory and then discard it after building.

You can find my script at this Github repository.

Note: when you modify the PATH with path_helper, your additions will be appended. The Python 3 installer prepends to the PATH. This might lead to slightly different behavior, as the Python 3 behavior overrides any system binaries. If you want to prepend for every user, you have to modify the /etc/paths file.

There are a few other simple installers where this approach makes sense. I also made a script that builds a package to create the .AppleSetupDone file in /var/db to suppress showing the setup assistant at first boot. Since I was planning to use this with the startosinstall --installpackage option, this script builds a product archive, rather than a flat component package.

You could create this package once and hold on to it whenever you need it again, but I seem to keep losing the pkg files. The script allows you to easily re-build the package in a different format or sign it when necessary. Also, dealing with the invisible file is a bit easier when you just create them on demand.

The last example creates a single invisible file .RunLanguageChooserToo, also in /var/db/. This will show an additional dialog before the Setup Assistant to choose the system language. MacAdmins might want to have this dialog for the obvious reason, but it also allows for a useful hack. When you invoke the Terminal at the Language Chooser with ctrl-option-command-T it will have root privileges, which allows some interesting workflows.

With this package the creation of the flag file happens too late to actually show the chooser. So I added the necessary PackageInfo flags to make the installer require a restart. Note that startosinstall will only respect this flag with a Mojave installer, not with High Sierra.

These three scripts can be used as templates for many similar use cases. As your needs get more complex, you should move to pkgbuild scripts with a payload folder, munkipkg, or Whitebox Packages.

You can learn about the details of inspecting and building packages in my book: “Packaging for Apple Administrators”