Category: Packaging

zsh scripts and Root Escalations

There was an update for the Support.app this week which fixed a CVE. There is an argument to be had about whether this CVE deserves its high rating, but it is worth discussing the underlying issue and presenting some solutions. (Erik Gomez has some great comments on Mac Admins Slack.)

zsh is far more configurable than most other shells, most certainly more configurable than the aging bash 3.2 that comes with macOS. One aspect of being more configurable is that it has multiple different configuration files that are loaded at different times when the shell starts. The most common configuration file you will have encountered is ~/.zshrc which is loaded for all interactive shells.

Another, less commonly used, configuration file, ~/.zshenv (and its global sibling /etc/zshenv) is loaded every time a zsh process launches, including script launches. That means, when you (or something else in the system) launches a script with a #!/bin/zsh shebang, the zsh process will read the files /etc/zshenv and ~/.zshenv when they exist, and execute their code.

Update 2024-03-22: some links for further reading:

The Setup

We can see this in action.

First, create a simple zsh “hello.sh” type script with this content:

#!/bin/zsh
echo "Hello, $(whoami)"

Make the script file executable (chmod +x hello.sh) and run it. You should see output like

% ./hello.sh
Hello, armin

where armin is replaced with your current username.

Next, create a new file ~/.zshenv (~/ means at the root of your home directory) with your favored text editor and add the following line:

echo "zshenv: as $(whoami) called from $0"

If you already have a ~/.zshenv you will want to rename it for now so we don’t modify that. (mv ~/.zshenv ~/.zshenv_old)

Note that the configuration file neither has a shebang, nor does it need to be executable.

When you open a new Terminal window, you should see the line

zshenv: as armin called from -zsh

among the other output at the top of the Terminal window, since ~/.zshenv is read and evaluated every time a zsh process starts. The shebang at the beginning of the script ensures a new zsh environment is created for it, even when your interactive shell is not zsh.

When you run hello.sh you will see that ~/.zshenv is executed as well:

% ./hello.sh
zshenv: as armin called from /bin/zsh
Hello, armin

So far, so good. This is how zshenv is supposed to work. It’s purpose is to contain environment variables and other settings that apply to all processes on the system. On macOS this is undermined by the fact that most apps and process are not started from a shell, so they don’t see environment variables set anywhere in the zsh (or another shell’s) configuration files, so .zshenv is rarely used. .zshrc is far more useful.

But now we come to escalation aspect: run hello.sh with root privileges using sudo:

% sudo ./hello.sh
zshenv: as root called from /bin/zsh
Hello, root

We see that both our script and ~/.zshenv are run with root privileges. Again, that is how sudo and .zshenv are supposed to work.

Since the zshenv configuration files are evaluated every time a zsh is launched, it is far more likely that they will eventually run with elevated privileges, than the other configuration files. Other shells, like bash and sh have no equivalent configuration file that gets run on every launch.

The Escalation

The potential danger is the following: a process only needs user privileges to create or modify ~/.zshenv. A malicious attacker can inject code into your .zshenv, wait patiently for a script with a zsh shebang to be run with root privileges, and then execute some malicious code with root privileges on your system.

This can also occur with preinstall or postinstall scripts with a zsh shebang in installation package files (pkg files) when the installation is initiated by the user. A malicious attacker could scan your system for apps where they know the installer package contains zsh scripts, which makes the chance that eventually a zsh script is run with root permissions quite certain. Then all they have to do is be patient and wait…

As root escalations go, this is not a very efficient one. The attacker is at the mercy of the user to wait when they execute a zsh script with escalated root privileges. On some Macs, that may very well be never.

This also only works when the user can gain administrative privileges. Standard users cannot use sudo to gain root privileges, or run installation packages.

There are more effective means of escalating from user to root privileges on macOS. Most easily by asking the user directly for the password with a dialog that appears benign. However, if you are a frequent user or author of zsh scripts, it is important to be aware.

From what I can tell so far, this does not affect scripts launched from non-user contexts, such as scripts from Munki, Jamf Pro or other management solutions, though there may be odd, unexpected edge cases.

As an organization, you can monitor changes to all shell configuration files with a security tool (such as Jamf Protect). There are more shenanigans an attacker could achieve by modifying these files. There are, however, many legit reasons to change these files, so most changes will not indicate an attack or intrusion. Nevertheless, the information could be an important puzzle piece in combination with other behaviors, when putting together the progress or pattern of an intrusion.

Update 2024-03-22: Matteo Bolognini has added a custom analytic for Jamf Protect to detect changes to the zshenv file.

Configuration file protection

This weakness is fairly straightforward to protect against. First, since the attack requires modification of ~/.zshenv, you can protect that file.

If you did not have a ~/.zshenv, create an empty file in its place or remove the line of code from our sample .zshenv before. Then apply the following flag:

% sudo chflags schg ~/.zshenv

This command sets the system immutable flag (schg, ‘system change’), which makes it require root privileges to modify, rename or delete the file.

If you want to later modify this file, you can unlock it with

% sudo chflags noschg ~/.zshenv

Just remember to protect it again when you are done.

Script protection

Since most Mac users will never open terminal and never touch any shell configuration file, relying on protecting the configuration file is not sufficient. However, it is quite easy to protect the zsh scripts you create from this kind of abuse.

One solution is to use a $!/bin/sh shebang instead of #!/bin/zsh. Posix sh does not have a configuration file which gets loaded on every launch, so this problem does not exist for sh.

If you are using zsh features that are not part of the POSIX sh standard you will need to change the script. In my opinion, installation scripts that are too complex to use POSIX sh, are attempting to do too much and should be simplified, anyway. So, this can also work as an indicator for “good” installation scripts.

In addition, installer packages might be used in situations where zsh is not available. Installation tools that install package files when the Mac is booted to Recovery (where there is no zsh) are used far less than they used to, but it is still possible and making your installer pkgs resilient to all use cases is generally a good choice.

I believe for installation package scripts, using an sh shebang is a good recommendation.

Some scripts popular with Mac Admins, like Installomator, do not usually run from installer packages, but are regularly run with root privileges. For these cases, (or for installation scripts, where cannot or do not want to use sh) you can protect from the above escalation, by adding the --no-rcs option to the shebang:

#!/bin/zsh --no-rcs
echo "Hello, $(whoami)"

zsh’s --no-rcs option suppresses the launch of user level configuration files. The rc stands for ‘Run Command’ files and is the same rc that appears in zshrc or bashrc or several other unix configuration files.

That allows you to keep using zsh and zsh features, but still have safe scripts. This is the solution that Support.app and the latest version of Nudge have implemented. I have also created a PR for Installomator, which should be merged in the next release.

Conclusion

Modifying ~/.zshenv is not a very effective means of gaining root privileges, but it is something developers and Mac admins that create zsh script that may be run with root privileges should be aware of. Switch to an sh shebang or add --no-rcs to the zsh shebang of scripts that might be run with root privileges to protect.

Build a notarized package with a Swift Package Manager executable

One of the most popular articles on this blog is “Notarize a Command Line Tool with notarytool.” In that post I introduced a workflow for Xcode to build and notarize a custom installer package for a command line tool. This workflow also works with apps and other projects that require a customized installer package building workflow. I use it in many of my own projects.

But Xcode is not the only way to build Swift binaries. Especially for command line tools, you can also use Swift Package Manager. This provides a mostly command line based interface to building and organizing your project, which you might prefer if you want to use an IDE that is not Xcode, or have Swift projects that need to run cross-platform.

I also have an older article on building a command line tool with Swift Package Manager. But then, I did not create an installer package or notarize the resulting binary.

Placing the binary in an installer package file is the best way to distribute a binary as you can control where in the file system the binary is installed. Notarizing the pkg file is necessary when you are distributing a command line tool, since it enables installations without scary dialogs or handling quarantine flags.

Also, some of the behavior of Swift Package Manager (SPM) and Xcode have changed since the previous posts. So, this article will introduce an updated workflow using Swift Package Manager tools and how to sign, package and notarize a command line tool for distribution.

Note on nomenclature: Swift Package Manager projects are called ‘packages.’ On macOS, installer files (with the pkg file extension) are also called ‘packages.’ We will be using SPM to build a notarized installation package (a pkg file) from a Swift package project. This is confusing. There is not much I can do about that other than using ‘installer package’ and ‘Swift package project’ to help distinguish.

Prerequisites

I wrote this article using Xcode 14.3.1 and Swift 5.8.1. It should also work with somewhat older or newer versions of Xcode and Swift, but I have not tested any explicitly.

Since I said earlier that using Swift Package Manager allows us to not use Xcode and maybe even build a cross-platform project, you may be wondering why we need Xcode. While we don’t need Xcode for our project, it is one way of installing all the tools we need, most importantly the swift and notarytool binaries. You get those from Developer Command Line tools, as well. We will also see that we can combine Xcode with the command line Swift Package Manager workflow, which I find a very useful setup.

To submit a binary to Apple’s notarization process you will need a Personal or Enterprise Apple Developer account, and access to the Developer ID Application and Developer ID Installer certificates from that account. A free Apple Developer account does not provide those certificates, but they are necessary for notarization

You can follow the instructions in the Xcode article on how to get the certificates and how to configure notarytool with an application specific password. If you had already done this previously you should be able to re-use all of that here. When you reach the ‘Preparing the Xcode Project’ section in that article, you can stop and continue here. Apple also has some documentation on how to configure notarytool.

The sample code we will be using will only work on macOS as it uses CoreFoundation functions. Installer packages and notarization are features of macOS, too, so this is not really a problem here. You can use this workflow to build macOS specific signed binaries and notarized installation pkg files from a cross-platform Swift package project. This will work as long as you keep in mind that the tools to sign, package and notarize only exist and/or work on macOS.

The sample code

We will build the same simple sample tool as in the last article. The prf command (short for ‘pref’ or ‘preference’) reads a default setting’s effective value using the CFPreferencesCopyAppValue function.

The macOS defaults command will read preferences, but only from the user level, or from a specified file path. This ignores one of the main features of macOS’ preferences system as it will not show if a value is being managed by a different preference level, such as the global domain, a file in /Library/Preferences, or (most importantly for MacAdmins) a configuration profile.

You can learn all about preferences and profiles in my book “Property Lists, Preferences and Profiles for Apple Administrators.”

We will build a really simple command line tool, named prf which shows the effective value of a setting, no matter where the value comes from. You could make this tool far more elaborate, but we will keep it simple, since the code is not the actual topic for this article.

We will also be using the Swift Argument Parser package to parse command line arguments and provide a help message. We could build this simple tool without using Argument Parser, but using an external package module is one of the strengths of using Swift Package Manager.

Create the Swift Package project

With all the preparations done, it is time to create our Swift package. We will do all the work in the shell, so open Terminal or your other favorite terminal app and navigate to the directory where you want to create the project.

> cd ~/Projects

Then create a new directory with the name swift-prf. This will contain all the files from the Swift package project. Change directory into that new directory. All following commands will assume this project directory is the current working directory.

> mkdir swift-prf
> cd swift-prf

Then run the swift tool to setup the template structure for our command line tool or ‘executable.’

> swift package init --type executable 
Creating executable package: swift-prf
Creating Package.swift
Creating .gitignore
Creating Sources/
Creating Sources/main.swift

You can inspect the hierarchy of files that the init tool created in the Finder (open .) or in your preferred editor or IDE.

.gitignore
Package.swift
Sources
    main.swift

`

You can open this package project in Xcode. In older versions of Xcode you had to run a special swift package command to generate the Xcode project, but now, Xcode can open Swift package projects directly. Use xed (the ‘Xcode text editor invocation tool’) to open the current directory in Xcode.

> xed .

There is a pre-filled .gitignore (which will be hidden in Finder and probably your IDE), a Package.swift, and a Sources directory with a single main.swift inside. If you want to use git (or another version control) system, now is the time to initialize with git init.

Build the project with swift build and/or run it with swift run. Not surprisingly, the template prints Hello, world!.

> swift build
Building for debugging...
[3/3] Linking swift-prf
Build complete! (0.92s)
> swift run  
Building for debugging...
Build complete! (0.11s)
Hello, world!

After building, there will also be a .build directory (also hidden in Finder, unless you toggle the visibility of invisible files using shift-command-.) which contains all the interim files. In the debug folder, you can find the swift-prf executable. You can run it directly:

> .build/debug/swift-prf
Hello, world!

You can clean all the generated pieces from the .build directory with swift package clean. This will leave some empty folders behind but remove all the interim and final products. This means the next build is going to take much longer, but this can be helpful after reconfiguring the Package.swift file or when the compiler gets confused.

Sidenote: when you use Xcode to edit your Swift package project, and choose Build or Run from the Xcode interface, then it will build and run in a different location (~/Library/Developer/Xcode/DerivedData/swift-prf-<random-letters>/Build). You need to be aware of this when you alternate between Xcode and the command line.

Configuring the Package

The Package.swift file contains the configuration for a Swift package project. You can see that the executable package template has a single target named swift-prf that builds from the files in Sources.

To change the name of the executable file, change the value of the name: of the .executableTarget to just prf. There is another name: earlier in the file, that sets the name of the entire project, you can leave that being swift-prf. They do not have to match.

Then build the project in the command line and run it directly:

> swift build
Building for debugging...
[3/3] Linking prf
Build complete! (0.51s)
> .build/debug/prf          
Hello, world!

We want to add the Swift Argument Parser package to our project as a dependency, so we can use its functionality in our code. For that, we will have to add a ‘dependency’ to the project and then to the target, as well. Modify the Package.swift file to match this:

// swift-tools-version: 5.8
// The swift-tools-version declares the minimum version of Swift required to build this package.
import PackageDescription
let package = Package(
  name: "swift-prf",
  products: [
    .executable(name: "prf", targets: ["prf"]),
  ],
  dependencies: [
    .package(url: "https://github.com/apple/swift-argument-parser", from: "1.2.0"),
  ],
  targets: [
    .executableTarget(
      name: "prf",
      dependencies: [.product(name: "ArgumentParser", package: "swift-argument-parser")],
      path: "Sources")
  ]
)

This means that our project uses the package available at the given URL, and our target is going to use the specific product (or module or framework) named ArgumentParser from that package. Some packages have several products combined out of several targets.

You can find more information on the format of the Package.swift file in this overview, and the full documentation.

The next time you build after this change, it will download the repository, build and link to toward your executable. That might take a while. The next build should be much faster again. Also, a Package.resolved file will appear in the project. This file caches the current versions of the included packages protecting you from unexpected changes when a package repo dependency updates. You can force Swift Package Manager to update the file with swift package update.

Sprinkle some code

Now that we have the Swift package project prepared, we can add the code to actually do something.

First, let’s keep the ‘Hello, world!’ for a moment, but put it in the right struct to use ArgumentParser. Change main.swift to:

import Foundation
import ArgumentParser
@main
struct PRF: ParsableCommand {
  func run() {
    print("Hello, world!")
  }
}

This should build and run fine from the command line with swift build and swift run. However, when you open this now in Xcode, you will see an error: 'main' attribute cannot be used in a module that contains top-level code

This comes from a long-running issue in Swift. In older versions of Swift it appears on the command line, as well. The work-around is easy though. It only seems to appear when the @main designator is the main.swift file. We can rename our main file to PRF.swift.

You may want to close the Xcode project window before you do this because this can confuse Xcode. If you manage to get Xcode into a confused state where the project in Xcode does not match what is on disk any more, quit Xcode and delete the .swiftpm/xcode directory, which is where Xcode keeps its generated files.

> mv Sources/main.swift Sources/PRF.swift

Now the project should build and run the same with the Swift Package Manager tools and in Xcode.

Now we can add the ‘full’ code for our tool. Keep in mind that the goal of this tutorial is not to learn how to write complex swift code for command line tools, but to learn the infrastructure requires to create and distribute them, so this code is intentionally simple and basic.

import Foundation
import ArgumentParser
@main
struct PRF: ParsableCommand {
  static var configuration = CommandConfiguration(
    commandName: "prf",
    abstract: "read effective preference value",
    version: "1.0"
  )
  @Argument(help: "the preference domain, e.g. 'com.apple.dock'")
  var domain: String
  @Argument(help: "the preference key, e.g. 'orientation'")
  var key: String
  func run() {
    let plist = CFPreferencesCopyAppValue(key as CFString, domain as CFString)
    print(plist?.description ?? "<no value>")
  }
}

When you compare that to the code from the last article, there are a few differences. We are using the @main attribute to designate the main entry point for the code (this was added in Swift 5.3) and I have added some help text to the tool and argument declarations.

When you use Swift Argument Parser, you should study the documentation on adding help to [commands](I have added some help text to the tool and argument declarations. ) and flags, arguments and options. (To be honest, you should read the entire documentation, a lot has changed since the last article.)

When you now run the tool:

> swift run  
Building for debugging...
[3/3] Linking prf
Build complete! (0.54s)
Error: Missing expected argument '<domain>'
OVERVIEW: read effective preference value
USAGE: prf <domain> <key>
ARGUMENTS:
  <domain>                the preference domain, e.g. 'com.apple.dock'
  <key>                   the preference key, e.g. 'orientation'
OPTIONS:
  --version               Show the version.
  -h, --help              Show help information.

We get the help text generated by Swift Argument Parser with the extra information we provided in the code.

If you want to provide the arguments to the swift run you have to add the executable name, as well:

> swift run prf com.apple.dock orientation       
Building for debugging...
Build complete! (0.11s)
left

Or you can run the executable directly from the .build/debug directory. (This will not automatically re-build the command like swift run does.

> .build/debug/prf com.apple.dock orientation
left

Since we provided a version in the CommandConfiguration, ArgumentParser automatically generates a --version option:

> .build/debug/prf --version       
1.0

Now that we have a simple but working tool, we can tackle the main part: we will package and notarize the tool for distribution.

Preparing the binary

When you run swift build or swift run it will compile the tool in a form that is optimized for debugging. This is not the form you want to distribute the binary in. Also, we want to compile the release binary as a ‘universal’ binary, which means it will contain the code for both Intel and Apple silicon, no matter which CPU architecture we are building this on.

The command to build a universal release binary is

> swift build --configuration release --arch arm64 --arch x86_64

When that command is finished, you will find the universal binary file in .build/apple/Products/Release/prf. we can check that it contains the Intel (x86_64) and Apple silicon (arm64) with the lipo tool:

> lipo -info .build/apple/Products/Release/prf
Architectures in the fat file: .build/apple/Products/Release/prf are: x86_64 arm64

For comparison, the debug version of the binary only contains the platform you are currently on:

> lipo -info .build/debug/prf
Non-fat file: .build/debug/prf is architecture: arm64

Apple’s notarization process requires submitted binaries to fulfill a few restrictions. They need a timestamped signature with a valid Developer ID and have the ‘hardened runtime’ enabled.

Xcode will always sign code it generates, but the swift command line tool does not. We will have to sign it ourselves using the codesign tool. You will need the full name of your “Developer ID Application” certificate for this. (Don’t confuse it with the “Developer ID Installer” certificate, which we will need later.)

You can list the available certs with

> security find-identity -p basic -v

and copy the entire name (including the quotes) of your certificate. Then run codesign:

> codesign --sign "Developer ID Application: Your Name (ABCDEFGHJK)" --options runtime  --timestamp .build/apple/Products/Release/prf

You can verify the code signature with

> codesign --display --verbose .build/apple/Products/Release/prf

Build the installation package

Now that we have prepared the binary for distribution, we can wrap it in an package installer file.

To cover all deployment scenarios, we will create a signed ‘product archive.’ You can watch my MacDevOps presentation “The Encyclopedia of Packages” for all the gory details.

First, create a directory that will contain all the files we want put in the pkg. Then we copy the binary there.

> mkdir .build/pkgroot
> cp .build/apple/Products/Release/prf .build/pkgroot/

Then build a component pkg from the pkgroot:

> pkgbuild --root .build/pkgroot --identifier com.scriptingosx.prf --version 1.0 --install-location /usr/local/bin/ prf.pkg

The --identifier uses the common reverse domain notation. This is what the installer system on macOS uses to determine whether an installation is an upgrade, so you really need to pay attention to keep using the same identifier across different versions of the tool. The --version value should change on every update.

The --install-location determines where the contents of the payload (i.e. the contents of the pkgroot directory) get installed to. /usr/local/bin/ is a useful default for macOS, but you can choose other locations here.

Next, we need to wrap the component pkg inside a distribution package.

> productbuild --package prf.pkg --identifier com.scriptingosx.prf --version 1.0 --sign "Developer ID Installer: Your Name (ABCDEFGHJK)" prf-1.0.pkg

It is important that you use the “Developer ID Installer” certificate here. The --identifier and --version are optional with productbuild but this data required for some (admittedly rare) deployment scenarios, and we want to cover them all.

You can inspect the installer pkg file with a package inspection tool such as the amazing Suspicious Package. The package file should as a signed “Product Archive.”

We don’t need the component pkg anymore, and it’s presence might be confusing, so let’s remove it:

> rm prf.pkg

Note: If you want to learn more about building installation packages, check out my book “Packaging for Apple Administrators”

Notarization

We are nearly there, just two more steps.

It is important to notarize pkgs that will be installed by a user, because otherwise they will get a scary warning that Apple can’t verify the pkg for malicious software.

notarytool submits the installer package to Apple’s Notarization process and returns the results. Use the keychain profile name you set up, following the instructions in the previous article or the instructions from the Apple Developer page.

> xcrun notarytool submit prf-1.0.pkg --keychain-profile notary-example.com --wait

This will print a lot of logging, most of which is self-explanatory. The process might stall at the “waiting” step for a while, depending on how busy Apple’s servers are. You should eventually get status: Accepted.

If you got a different status, or if you are curious, you can get more detail about the process, including rejection reasons, with notarytool log. You will need the ‘Submission ID’ from the submission output:

xcrun notarytool log <submission-uuid> --keychain-profile notary-example.com

As the final step, you should ‘staple’ the notarization ticket to the pkg. This means that the (signed) notarization information is attached to the pkg-file, saving a round trip to Apple’s servers to verify the notarization status when a system evaluates the downloaded installer package file.

xcrun stapler staple prf-1.0.pkg
Processing: /Users/armin/Desktop/swift-prf/prf-1.0.pkg
Processing: /Users/armin/Desktop/swift-prf/prf-1.0.pkg
The staple and validate action worked!

And with that, we have a signed and notarized installation pkg file! You can verify this with spctl:

> spctl --assess --verbose -t install prf-1.0.pkg 
prf-1.0.pkg: accepted
source=Notarized Developer ID

Automation

While it is instructive to do this process manually, it is also quite complex and error-prone. If you have been following this blog for any time, you will know that I don’t stop at detailed step-by-step instructions with explanations.

You can find a script to automate all of these steps here. The enclosing repository includes the entire project (all three files) for your reference.

There is a section at the beginning with variables to modify with the information specific to your environment and project, such as your developer ID information and the name of the credential profile for notarytool. Then there are a few variables, such as the product name, and the installation package identifier.

Run the pkgAndNotarize.sh script from the root of the Swift package project directory.

./pkgAndNotarize.sh

The script creates the installer pkg file in the .build directory. The last line of output is the path to the final, signed, notarized and stapled pkg file.

The script mostly follows the process described above, with a few extras. For example, the script determines the version dynamically by running the tool with the --version option. It also uses the modern compression options I described in this post.

If any of the steps in the script fail, you can determine what exactly failed from the output, error message and error code.

(I belief that this could probably be a makefile, but I have no experience with that (yet). I guess I will need to ‘make’ time for this…)

Conclusion

Apple provides developers and MacAdmins with amazing platforms and tools to build all kinds of powerful apps, tools and automations. But then they don’t really document any of the processes or best practices at all. The amount of searching, guesswork, and frustrating trial and error necessary to piece all of this together for a workflow like this one is quite the shocking condemnation of Apple’s documentation.

There are glimmers of hope. The documentation for the notarization process and notarytool are exemplary.

But they only cover one piece of this puzzle. A developer building a tool has to still figure out how to

  • sign all the binaries properly
  • assemble the binaries and resources necessary into an installation package payload
  • how (and when, and when not) to use pre- and postinstall scripts
  • which kind of package installer to build and which tools to use
  • securely manage the Developer ID certificates (this is especially challenging for developer teams)
  • automate this workflow with Xcode or Swift Package Manager or a CI/CD system

MacAdmins often complain about poorly built installer pkgs, and often for good reasons. But to be fair, there are no best practices and little to no documentation for this from Apple. How are developers supposed to know all of this? Most MacAdmins can define what an installer package should do and not do, but wouldn’t be able to explain to a developer how to build such an installer package, let alone integrate that into their build automations. And most developers don’t even know a MacAdmin to ask about this.

Apple requires that developers create signed and notarized archives for software distribution. And I agree wholeheartedly with their motivations and goals here. But when impose requirements for distribution, you have to make the process of creating the installers the correct way easy, or at least well documented, whether you use Xcode or a different tool set, whether you want to distribute a simple self-contained app, a single command line tool, or a complex suite of tools and resources.

Apple has their work cut out to improve this. Official best practices and sample workflows for installer creation and distribution that consider and respect the requirements of MacAdmins for deployment, have been disgracefully lacking for a long time. The more requirements and security Apple piles on to application and tool distribution, the more desperately they need to provide documentation, best practices and guidance.

Until that happens, you have my paltry scripts.

Update: quickpkg 1.0beta

QuickPkg is one of those tools that I built for myself a long time ago and has remained useful (at least to me) without many updates.

Since Apple will be removing the pre-installed Python 2 binary from macOS 12.3, I was forced to pull the code out, dust it off and update it to Python 3. I chose to use the MacAdmins “Managed Python” since it provides a nice bundle of libraries which should be useful for some other tools I have (and still need to migrate). It should (probably) work with other Python 3 distributions, as well, but I did not test this at all.

Thankfully, quickpkg did not require very many code changes. I could even remove all the code from Greg Neagle’s FoundationPlist. Python 2 plistlib could not read binary plist files, so that extra functionality was required. But Python 3 plistlib can read and write binary plists and I could save a lot of code.

The script has worked with all of my tests, but I probably have not explored all the weird edge cases yet. So I am releasing it as a beta. Please file issues and/or pull requests on the GitHub repo if you run into any problems.

Update: pkgcheck

The macOS Monterey 12.3 beta release notes say that the Python 2.7 binary (located in /usr/bin/python) will be removed. Since you follow this blog, this should not come as a surprise. We have been warned about this since Catalina. (Or longer)

That said, the removal of Python 2 in a minor macOS release is surprising. Minor updates should not have breaking changes or removals. Admins and developers may not expect removals and other breaking changes in a minor update and therefore not be paying as much attention to changes. Also, the time a minor update is in beta is usually 6-8 weeks, which leaves us and developers much less time to find and fix problems than a major update beta phase, which is usually 4-5 months.

Nevertheless, we have to work with what Apple deals to us. MacAdmins have been investigating their own tools and scripts since the Monterey release or earlier to avoid the prompts. But when you get vendor pkgs, these might contain anything. While you can inspect pkgs with tools like pkgutil, Pacifist or Suspicious Package, it can get tedious with many packages.

A while back I built a script called pkgcheck to automate this check. Since I (and many others) have started using it again in the recent days, I have added a few more checks to it.

The earlier version would flag files in the installer’s resources that had a /bin/bash, /usr/bin/python, /usr/bin/ruby, or /usr/bin/perl shebang. (the first line with the #!) I have now also added check for a shebang with /usr/bin/env [python|ruby|perl] because when run from an installer pkg, this will also resolve tousing the built-in, deprecated runtimes. Also, using python in the shebang will now be shown as a red error, rather than a yellow warning.

The script will now also grep for use of python in installation scripts and show those scripts. This might generate a few false positives. You will have to use your judgement. For example using python3 in an installation script will also trigger this. But then, it probably should, since python3 is not installed on macOS by default. (What you see in /usr/bin/python3 is a shim that prompts you to install the Command Line Developer Tools, unless they or Xcode are already installed.)

I hope this is useful!

Suspicious Package 4.0 Update

I recently posted about some Suspicious Package Power User Features, which was a follow-up to my MacDevOps YVR presentation “The Encyclopedia of Packages.”

As a follow-up to that follow-up, Suspicious Package was updated to version 4.0 today. (Yesterday? Time zones are strange.) The update to brings compatibility with macOS Monterey and some really nice refinements to these power user features.

First and foremost, Suspicious Package will now show the kind of package, or “package format” in the Package Info tab. This makes me very happy, not just because the FAQ references my presentation. As the FAQ correctly states, most users of packages or even the Suspicious Package application will not care much about the differences between the package formats, but for system administrators, this can determine the difference between a functional deployment or a broken workflow.

The previously ‘secret’ option to show the PackageInfo xml file is now also exposed in the preferences window, next to the option to show the Distribution XML.

It is now also easier to search for the contents of a particular component in a distribution package.

You can download the latest version of Suspicious Package and get the update notes here.

Many thanks to Randy Saldinger of Mothers Ruin Software for providing this amazing tool and further refining it!

Randy was also recently a guest on the MacAdmins Podcast. You check it out if you have not yet listened to that episode.

Suspicious Package Power User Features

As many MacAdmins, I work a lot with installer packages. You can say I wrote the book about it. When you get an installer package from some vendor website, you will want to inspect it before you install it anywhere, let alone deploy it to dozens, hundreds, or even thousands of Macs in your fleet.

You can use the pkgutil tool to do this in the command line, but there are package inspector tools with a graphical interface that are very useful and popular.

One of these tools is Suspicious Package from Mothers Ruin Software. It displays all of the content and resources of a pkg file in a very nice user interface. Many people love Suspicious Package from Mothers Ruin Software. I have always had reservations about Suspicious Package, though, because I thought there were a few missing features.

Update (August 2021): The app has gotten a new major update (4.0) which makes these “power user features” even more accessible!

The missing features were in connection with distinguishing “normal” or “component” packages (which have a single payload) from “distribution” or “meta” packages (which don’t have a payload of their own, but contain one or more component packages).

I have explained the differences in a bit more detail in my MacDevOps YVR presentation “The Encyclopedia of Packages” where I (once again, ignorantly) stated that you can’t really tell them apart in the UI of Suspicious Package.

After that presentation, Mat X, one of the organizers of the conference got me in touch with the developer of Suspicious Package, Randy Saldinger, who graciously and patiently demonstrated that I was wrong.

In my defense, you really cannot tell normal packages from distribution packages in the default configuration of Suspicious Package, but if I had bothered to read the manual and/or explore the Preferences window, I would have found this option:

This will show the Distribution xml file at the top of the list of the ‘All Scripts’ pane for distribution packages. When you see no Distribution file there, the package is a component package.

The second checkmark in that preference window is also very useful. With “Component package and bundle info” enabled you can see which component contains the selected file in the info pane:

You can also search in the “All Files” tab with command-F and use the component package ID as a search criteria.

All of this is already well enough to remove the reservations I have had on Suspicious Package. But Randy shared another preference with me which puts it over the top. It is not exposed in the UI (yet) but when you run:

% defaults write com.mothersruin.SuspiciousPackage ShowRawPackageInfo -bool YES

in Terminal and re-launch Suspicious Package, you will see the raw PackageInfo xml in the “All Scripts” tab. Together with showing the Distribution xml, this allows you to inspect all the raw metadata that can be inside a pkg file.

I have also learned that you can use the search functionality in the “Help” menu of Suspicious Package which will link directly to the online documentation. Not many apps leverage this functionality, so we often forget to check for it. Kudos to Suspicious Package for using this.

Many thanks to Randy for all his work and help and for providing an excellent tool! I am very much looking forward to the next version.

Platform Support in macOS Installer Packages (pkg)

Mac users and admins find themselves in yet another major platform transistion. For the duration of the transition, developers and admins will have to deal with and support software and hardware for the Intel and Apple silicon Macs. With Universal applications and Rosetta 2, Apple is providing very efficient tools to dramatically reduce the friction and problems involved.

This post was inspired by comments from Josh Wisenbaker on MacAdmins Slack and Twitter. Thank you!

For most end user level tasks, these tools will provide seamless experience. Universal applications will run on either platform natively and Rosetta 2 will translate applications compiled for the legacy platform (Intel) so they can run on the new Apple silicon chips. There are only a few situations where these tools don’t work: virtualization solutions and Kernel extensions.

In most cases this tools will “just work.” But for MacAdmins there is one major issue that may throw a wrench in your well-oiled deployment workflows. Rosetta is not pre-installed on a fresh macOS installation.

We can only speculate why Apple chooses to deliver Rosetta this way. In “normal” unmanaged installations, this is not a big deal. The first time a user installs or launches a solution that requires Rosetta, they will be prompted to for installation and upon approval, the system will download and install Rosetta.

As a MacAdmin, however, you want your deployments to be uninterrupted by such dialogs. Not only are they confusing to end users, but the user might cancel out of them which will result in your workflow failing partially.

There are two solutions. The first is to install Rosetta as early as possible in the deployment process. Apple provides a new option for the softwareupdate command to initiate the installation. Graham Gilbert and Rich Trouton have already published scripts around this. Have this script run early in your deployment workflow on Apple silicon and subsequent apps and tools that require Rosetta should be fine.

The other solution is to avoid requiring Rosetta and thus the prompt for Rosetta.

I mentioned earlier that we can only speculate as to why Apple has made Rosetta 2 an optional installation. One possible explanation is, that Apple believes Rosetta will not be a necessary installation for very long. An extra dialog and installation will make users and developers more aware of software that “needs an update” and motivate developers to provide Universal applications faster.

When a user opens an application that requires Rosetta for the first time, before Rosetta is installed, the system prompts to install. The same thing can happen with an installer package. The system might prompt to install Rosetta before a certain package is installed. However, not all packages trigger the dialog. I was curious what is required in the package to trigger or to avoid the prompt.

Aside from legacy formats, there are two types of packages. The first are “plain” packages, which are also called component packages. These packages have a payload and can have pre- and postinstall scripts, but other than that, there is little metadata you can add to influence the installation workflow.

This is where “distribution packages” come in. Distribution packages do not have a payload or installation scripts of their own, but contain one or more component packages. In addition, distribution packages can contain metadata that influences the installation workflow, such as customization of the Installer.app interface, system version checks, prompting the user to quit running applications before an installation and software requirements and a few more.

Note: learn more about the detailed differences between component and distribution packages in my book: “Packaging for Apple Administrators“

You can build a distribution package from a component package with the productbuild command:

> productbuild --package component.pkg distribution.pkg

Since most of the extra features of distribution packages are only effective when the installation package is launched manually in the Installer application, MacAdmins usually just build component pkgs.

The confusing part here is that both component pkgs and distribution pkgs have the same file extension. They are hard to distinguish even from the command line. To tell them apart, you can expand a pkg with the pkgutil command and look at the files in the expanded folder. Component pkgs have (among other files) a PackageInfo file and distribution pkgs have a Distribution file:

# component pkg
> pkgutil --expand component.pkg expanded_component_pkg
> ls expanded_component_pkg
Bom
Payload
Scripts
PackageInfo

# distribution pkg
> pkgutil --expand distribution.pkg expanded_distribution_pkg
> ls expanded_distribution_pkg
component.pkg
Distribution

For distribution pkgs, the Distribution file is an XML file which contains the configuration data for the package. One tag in this XML is the options tag which can have a hostArchitectures attribute. According to [Apple’s documentation on this tag](A comma-separated list of supported architecture codes), the hostArchitectures are a “comma-separated list of supported architecture codes.”

Apple documentation is a bit aged, so it gives i386, x86_64, and ppc as possible values. However, when you read the productbuild man page on macOS Big Sur you will see that arm64 is a new valid value. We will also find these extremely helpful note:

NOTE: On Apple Silicon, the macOS Installer will evaluate the product’s distribution under Rosetta 2 unless the arch key includes the arm64 architecture specifier. Some distribution properties may be evaluated differently between Rosetta 2 and native execution, such as the predicate specified by the sysctl-requirements key. If the distribution is evaluated under Rosetta 2, any package scripts inside of product will be executed with Rosetta 2 at install time.

When a distribution pkg has this attribute and it contains a value of arm64 then the installation process on an Apple silicon Mac will not check if Rosetta is installed. When arm64 is missing from the hostArchitectures, or the attribute or tag are missing entirely, the installation process on an Apple silicon Mac will asume the pkg requires Rosetta and prompt to install when necessary.

There is more good news in the next note in the man page:

NOTE: Starting on macOS 11.0 (Big Sur), productbuild will automatically specify support for both arm64 and x86_64 unless a custom value for arch is provided.

When you use productbuild to create a distribution pkg on Big Sur (Intel and Apple silicon) both arm64 and x86_64 will be added to the configuration by default.

But, when you use productbuild on Catalina or earlier, the attribute will be lacking, when means that when someone installs that pkg on an Apple silicon Mac, it will assume it requires Rosetta and prompt for installation.

Adding both architectures by default is a useful default. But can we set the value explicitly when we build the distribution pkg? And can we do so on Catalina?

Yes, you can, of course. There are even two solutions. First, instead of letting productbuild generate the Distribution xml, you can build and provide a complete Distribution xml file with the --distribution option. That will give you full, fine-grained control over all the options.

The second solution is a bit easier. You can create a requirements.plist property plist file in the form:

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
        <key>arch</key>
        <array>
                <string>x86_64</string>
                <string>arm64</string>
        </array>
</dict>
</plist>

Then you can provide this property list file to the productbuild command with the --product option.

> productbuild --package component.pkg --product requirements.plist distribution.pkg

This way, productbuild still generates the Distribution xml and merges in your choices from the requirements.plst. There are other options you can add which are documented in the productbuild man page.

Both of these approaches will work on Catalina as well. This way you can explicitly tell the installer system which architectures your packages will run with and not leave anything to chance.

In Whitebox Packages you can configure the hostArchitectures attribute under the “advanced options” for a distribution package.

As far as I can tell, when you install a component pkg, no checks for Rosetta are performed. Nevertheless, this is not something I would rely on. For packages that are crucial to the deployment workflow, I would recommend going the extra step and creating a distribution pkg from the component pkg with the proper flags set. This way you can ensure proper behavior.

Of course, if your package installer contains any form of Intel-only, not-universal binary, you should not abuse this just to skip the annoying Rosetta dialog, as it might lead to problems later. But, when the software you are installing is universal, you sould use this to tell the system which platforms your package supports.

Strategies to using desktoppr

A while back I introduced desktoppr. It is a very simple tool; its singular goal is to set the desktop picture (users and admins migrating from Windows call it ‘wallpaper,’ but that is not the proper macOS nomenclature) without requiring PPPC/TCC whitelisting.

The good news is that desktoppr still works fine, nearly one-and-a-half years in! Even though Catalina brought in many changes and restrictions, desktoppr kept doing its job.

Nevertheless, as I have used desktoppr myself in several different deployments, and I have a few different approaches to deployment, depending on the requirements.

Catalina split system volume

One of the new features of Catalina is a read-only system volume, separate from the data volume. This means that the pre-installed desktop pictures are no longer in /Library/Desktop Pictures/ but can now be found in /System/Library/Desktop Pictures. This is on the read-only part of the file system.

On a new “fresh” macOS installation, the /Library/Desktop Pictures does not exist. However, when you create this folder, its contents will appear in the ‘Desktop’ preference pane, merged with the contents of the protected system folder. So, we can continue to use /Library/Desktop Pictures as a place to store and install custom desktop image files.

Note: if you do not want the custom desktop picture to appear in the Desktop preference pane, then you can install the file in a different location. /Users/Shared or /Library/MyOrganization/ are useful locations.

Packaging the custom picture

> mkdir -p BoringDesktop/payload

> cd BoringDesktop
> cp /path/to/BoringBlueDesktop.png payload

> pkgbuild --root payload --install-location "/Library/Desktop Pictures/" --identifier com.example.BoringDesktop --version 1 BoringDesktop.pkg


These command will create a payload folder, copy an image file (my example is BoringBlueDesktop.png) and build an installation pkg using the pkgbuild command.

If you want a more detailed explanation of this process, you can find it in my book: “Packaging for Apple Administrators”

Lock the Desktop

In classroom, lab, and kiosk settings, MacAdmins may want to set and lock the desktop picture. In this use case, you do not need desktoppr at all.

Use the above pkg to install the image file and then use your management system to push a configuration profile that sets and locks a Desktop Picture.

Many management systems will have the desktop picture controls hidden in the ‘Restrictions’ payload among many other settings. Please consult the documentation. You can also use this custom profile that only controls the desktop setting.

Preset the desktop, but the let user change it

This is the most common way MacAdmins will want to deploy is to pre-set a custom Desktop Picture but allow the user to change it later. This is what desktoppr was created for.

There are two approaches you can take to do this. Well, to be honest, there are way more, and all of them are valid, as long as they work. I should say: I will show two different approaches.

The modular approach

In this case you use your management system to install and run all the pieces:

  • install the custom desktop picture using the above pkg
  • install desktoppr using the pkg*
  • run a script that sets the desktop

* for 10.14.3 and earlier, desktoppr v0.2 requires the Swift 5 runtime support for command line tools to be installed.

The advantage of this approach is that we already did the first part earlier, and the desktoppr pkg can be downloaded from the git repo,. So, we already have two of the three parts.

For the script, there is a sample script in the repository, as well.

Note that this script has changed slightly since the last post. Originally, the script used launchctl asuser. The behavior of launchctl asuser seems to have changed somewhat in a recent update and I have switched the script to use sudo -u instead.

This approach can be used with Munki, Jamf, outset, and many other management solutions.

All in one package

The downside of the modular approach is that you have to manage three pieces (the image file, the desktoppr binary, and a script) in your management system. This can be especially problematic when you are not the actual administrator of the management system but more active in a ‘consulting role.’

In this case, I have found it easier to build a single package that does all the work. This is easier to hand over to another admin. It is also more flexible and can be used in even more situations. It is a bit more work to assemble, though.

First, you need the ‘ingredients:’

* for 10.14.3 and earlier, desktoppr v0.2 requires the Swift 5 runtime support for command line tools to be installed.

First, create a project folder with a payload folder inside. Then copy the necessary files into the right place:

> mkdir -p BoringDesktop/payload
> cd BoringDesktop


Copy the image file to payload:

> cp /path/to/BoringBlueDesktop.png payload


Create a scripts directory and copy the postinstall script to it:

> mkdir scripts
> cp path/to/desktoppr/examples/postinstall scripts


Expand the zip archive (in Finder) and copy the desktoppr binary into the scripts folder.

> cp path/to/build/usr/local/bin/desktoppr scripts


Your project folder should now look like this:

BoringDesktop Project FolderBoringDesktop Project Folder

You can then build the pkg with:

> pkgbuild --root payload --scripts scripts --install-location "/Library/Desktop Pictures/" --identifier com.example.BoringDesktop --version 2 BoringDesktop-2.pkg


Note the different version number, so the system can recognize this as a different pkg from the one you might have built earlier.

This form of the pkg does not actually install the desktoppr binary on the target system. When the pkg is created, the entire contents of the scripts folder will be archived into the pkg file. When the pkg is being installed, the contents will be expanded into a temporary directory. That means that the postinstall script can see the binary in the same director ‘next to it.’ This happens in lines 22–27 of the postinstall script.

After the installation is complete, the temporary files will be removed, so the postinstall script and the desktoppr binary will be removed automatically. You don’t need to worry about the cleanup.

Conclusion

Which approach works best depends on your specific deployment needs, your management setup and workflows and (not the least) your comfort with building scripts and packages.

Even when you have defined your deployment needs, there are multiple solutions on how to build and deploy a custom desktop picture. As long as they achieve the desired goal, there is no “best” solution.

You can earn more details about building packages in my book: “Packaging for Apple Administrators”

Notarization for MacAdmins

Apple introduced Notarization in macOS Mojave. Since its introduction Apple has kept increasing the use of notarization checks in macOS. For macOS Catalina, Apple has been very vocal saying that Notarization is a requirement for distribution of Applications outside of the Mac App Store.

This has left many MacAdmins confused and concerned. A large part of the work as a MacAdmin consists of (re-)packaging applications, configuration files and scripts so they can be distributed in an automated fashion through a management system, such as Jamf Pro, Munki, Fleetsmith, etc.

Do MacAdmins need to notarize all the package installers they create as well? Do MacAdmin need to obtain an Apple Developer ID? How should MacAdmins deal with notarized and non-notarized applications and installers from third parties?

This post is an attempt to clarify these topics. It’s complicated and long, bear with me…

Signed Applications

Apple’s operating systems use cryptographic signatures to verify the integrity and source of applications, plug-ins, extensions, and other binaries.

When an application, plug-in, extension, or other binary (from now on: “software”) is signed with a valid Apple Developer certificate, macOS (or iOS, tvOS, and watchOS) can verify that the software has not been changed or otherwise tampered with since it was signed. The signature can verify the source of the signature, i.e the individual Developer account or Developer team whose Developer identity was used to sign the software.

If the contents of the software were changed for some reason, the verification fails. The software can be change by accident or with malicious intent, for example to inject malicious code into an otherwise beneficial piece of software.

Since Apple issues the Developer IDs, they also have the option of revoking and blacklisting certificates. This usually happens when a Developer ID has been abused to distribute malware. The Malware Removal Tool or MRT is the part of the system that will identify and block or remove blacklisted software.

App Store Distribution

Applications distributed through Apple’s App Stores have to be signed with a valid Developer ID. A developer needs to have valid subscription ($99 per year for individuals and $299 for organizations) to obtain a certificate from Apple.

When a developer submits software to an App Store on any Apple system, the software will be reviewed by Apple to confirm whether it meets the various guidelines and rules. This includes a scan for malware.

App Store applications also have to be sandboxed, which means they can only access their own data (inside the “sandbox”) and not affect other applications, services, or files without certain “entitlements” and, in many cases, user approval.

App Store rules and regulations and sandbox limitations preclude many types of applications and utilities. On iOS, tvOS and watchOS, they are the only way for developers to distribute software to end users.

Apple provides a method for Enterprises and Organizations to distribute internal software directly without going through the App Store and App Store review. This should be limited to distribution to employees and members of the organization (such as students of a university or school). This method has infamously been abused by Facebook and other major companies which lead to Apple temporarily revoking their certificates. (We will not discuss Enterprise App Distribution in this post.)

There is also much criticism about how realistic Apple’s rules and guidelines are, how arbitrary the review process is, and whether the sandbox restrictions are useful or unnecessarily draconic. A lot of this criticism is valid, but I will ignore this topic in this post for the sake of simplicity and brevity.

Software downloaded from the App Store is automatically trusted by the system, since it underwent the review and its integrity and source can be verified using the signature. In the rare case that some malicious software was missed but the review process, Apple can revoke the Developer certificate or blacklist the software with the Malware Removal Tool.

Distribution outside of the Mac App Store: Gatekeeper and Quarantine

As mentioned before, iOS, tvOS, and watchOS applications have to distributed to end users through the App Store, be signed with a valid Developer ID and under go the review.

Because the Mac existed a long time before the Mac App Store, software vendors have many ways of distributing software. Originally software was sold and delivered on physical media (Floppy Disks, CDs, and DVDs), but we with the rise of the internet, users could simply download software from the developer’s or vendor’s website or other, sometimes dubious, sources.

Apple has (so far) accepted and acknowledged that these alternative means of software distribution and installation are necessary on macOS. To provide an additional layer of security for the end user in this use case, Apple introduced Gatekeeper in OS X 10.8 Mountain Lion.

When a user downloads a software installer or archive from the internet it is ‘quarantined.’ When the user attempts to install or launch the software for the first time, Gatekeeper will evaluate the software. There are many steps in this evaluation, and Howard Oakley explains the process in much detail in this post.

You can see the quarantine flag with the xattr command:

% xattr ~/Downloads/somefile.pkg
com.apple.macl
com.apple.metadata:kMDItemWhereFrom
com.apple.quarantine

You can delete the quarantine flag with xattr -d com.apple.quarantine path/to/file. Usually, there is no real need to.

The first step of the evaluation is verifying the software’s signature, Developer ID, integrity. When encountering an unsigned piece of software the user will be presented with a warning dialog.

Users with administrator privileges can bypass Gatekeeper by choosing “Open” from the context menu instead of double-clicking to open. Gatekeeper can be completely disabled with the spctl command, though this is not recommended.

The Developer signature provides a way to verify the source and integrity of a piece of software, but since the distribution happens outside of Apple’s control, a malicious developer could still put any form of malicious code in the signed software to keep Gatekeeper happy. As long as the malware avoids widespread detection it will look good to Gatekeeper and the end user. Even when the malware is detected by Apple and the Developer ID is revoked, it is not hard for a malicious developer to obtain or steal a new Developer ID and start over.

Enter Notarization

Apple needed another layer of security which could scan software for known malware and enforce a certain set of security rules on third party software, even when it is distributed outside of the Mac App Store.

Note: I find the effort Apple is putting in to Gatekeeper and Notarization quite encouraging. If Apple wanted to restrict macOS to “App Store only” distribution in the near future, this effort would not be necessary. This shows that Apple still acknowledges the important role that independent software distribution has for macOS.

To notarize software, a developer has to sign it with their Developer ID, and upload it to Apple using Xcode or the altool command. Then Apple notarization workflow will verify that the software fulfills certain code requirements and scans for certain malware. The exact details of what is considered malware are unknown. However, we do know that the process is fully automated and, unlike the App Store approval process, does not involve human reviewers.

If the software has passed the notarization process the result will be stored on Apple’s servers. When Gatekeeper on any Mac verifies the software it can confirm the notarization status from Apple’s servers. Alternatively, a developer can ‘staple’ a ‘ticket’ to the software, which allows Gatekeeper to confirm the notarization status without needing to connect to Apple.

When Gatekeeper encounters a quarantined software that is notarized, it will show the familiar Gatekeeper dialog with an additional note that:

“Apple checked [the software] for malicious software and none was detected.”

Since 10.14.5, When Gatekeeper encounters signed software that is not notarized it will show the standard dialog but with an additional yellow warning sign.

As with the previous Gatekeeper checks for a valid signature an administrator user can override the check by choosing ‘Open’ from the context menu instead of double-clicking to open.

In Mojave notarization was enforced in Gatekeeper checks for kernel extensions and in 10.14.5 for software with new Developer IDs, which where created after June 2019.

Starting with Catalina, all software needs to be notarized to pass Gatekeeper when the first launch or installation is initiated by a user.

However, the warning can still be overridden by an administrator user using the context menu.

What can be Notarized

As of now, the following pieces of software can be notarized:

  • Application bundles
  • Kernel Extensions
  • Installer Packages (pkg), Disk images (dmg) and zip archives

When you are building other types of software, such as command line tools, you can (and should) place them in one of the archive formats. The preferred choice for MacAdmins should be an installer package (pkg) since it will also place the binary in the correct location in the file system with the correct access privileges.

What cannot be Notarized

You should not notarize a binary or application that you did not sign! The Developer ID used to sign a binary (application or command line tool) should be the same as the Developer ID used to submit the software for notarization.

Apple has loosened the requirements for notarization until Jan 2020 to give developers some extra time to adapt. Once the requirements return to the full restrictions an attempt to notarize third party software with a different Developer ID will fail. (Existing notarizations will remain valid after that date.)

Installer command

When you install software using the installer command from the Terminal or a script, it will bypass quarantine and the Gatekeeper check.

This is also true when you install software using a management system such as Jamf Pro, Munki, Fleetsmith, etc.

Software you re-package as a MacAdmin for distribution through management systems does not need to be notarized.

Given this and the limitations on notarizing third party software above, you should very rarely need to notarize as a MacAdmin.

Example: Re-packaging third party software from dmg

A lot of applications for macOS are distributed as disk images. The normal end user workflow would be to mount the dmg after downloading, and then copying the application from the dmg to the /Applications folder.

There are two steps where Gatekeeper might trigger: when you mount the disk image and when you launch the application after copying for the first time. To pass both these checks, a developer should prudently notarize both the disk image and the application. Google Chrome for example does exactly that, avoiding the Gatekeeper warning.

We can verify this with the spctl command:

% spctl -a -vv -t install ~/Downloads/googlechrome.dmg
/Users/armin/Downloads/googlechrome.dmg: accepted
source=Notarized Developer ID
origin=Developer ID Application: Google, Inc. (EQHXZ8M8AV)

% spctl -a -vv -t execute "/Volumes/Google Chrome/Google Chrome.app"
/Volumes/Google Chrome/Google Chrome.app: accepted
source=Notarized Developer ID
origin=Developer ID Application: Google, Inc. (EQHXZ8M8AV)

Unfortunately, some management systems don’t understand “Apps in disk images” as a distribution method. For these systems MacAdmins need to re-package the application into a pkg. You can do that quickly with pkgbuild:

% pkgbuild --component /Volumes/Google\ Chrome/Google\ Chrome.app --install-location /Applications/ GoogleChrome.pkg
pkgbuild: Adding component at /Volumes/Google Chrome/Google Chrome.app
pkgbuild: Wrote package to GoogleChrome.pkg

or use quickpkg.

This new installer package will be neither signed nor notarized:

% spctl -a -vv -t install GoogleChrome.pkg                          
GoogleChrome.pkg: rejected
source=no usable signature

When you send this installer package to another Mac with AirDrop, the receiving system will attach the quarantine flag. And when you double click it, you will get the Gatekeeper warning. However, when you can still install it using the installer command in Terminal, which bypasses the Gatekeeper system, just as your management system will:

% installer -pkg ~/Downloads/GoogleChrome.pkg -tgt /

Alternatively, you can choose “Open” from the context menu in Finder to bypass Gatekeeper. However, this is not something you want to teach your end users to do regularly.

Firefox can be downloaded as a disk image as well as a installer package. While the application inside both is notarized, neither the disk image nor the installer package are. The disk image mounts with no issues, but when you try to open the installer pkg by double-clicking you will get the expected notarization warning. Nevertheless, the pkg will work fine after importing to your management system.

Edge cases

There are some cases where notarization would be useful for MacAdmins but might not even be possible. I met a MacAdmin working at a university at MacSysAdmin last week. They need to re-package a VPN client with customized configuration files to be installed on student-owned machines.

There is really no solution without the students running into the notarization warning. Teaching the users how to bypass Gatekeeper is not a good solution.

In these cases you have to work with the software vendor and Apple to find a workable solution.

Summary

Notarization is a new security layer introduced by Apple in Mojave. The restrictions imposed on non-notarized software increase in Catalina.

When an Application is installed or launched for the first time by the user (by double-clicking) Gatekeeper will verify the signature and notarization status and warn the user if any are missing.

Developers should sign and notarize their applications and tools.

Mac Administrators should not notarize applications and tools from third parties.

Applications and packages installed through management systems bypass Gatekeeper and do not need to be notarized.

Conclusion

Apple is loudly messaging that notarization is absolutely required for applications in Catalina. While this message makes sense for the developers building the software, it does not apply to administrator who re-package third party software for distribution through management systems.

MacAdmins should join Apple in demanding signed and notarized binaries and installer packages from developers.

However, MacAdmins can also continue their current workflows for re-packaging and distribution.

Links

Check Installer Pkgs for deprecated scripts

macOS 10.15 Catalina will deprecate the built-in /bin/bash. I have talked about this at length.

The release notes for Catalina also tell us that other built-in scripting runtimes, namely Python, Perl, and Ruby. Will not be included in future macOS releases (post-Catalina) any more.

This means, that if you want to use bash, Python, Perl, or Ruby on macOS, you will have to install, and maintain your own version in the future.

However, scripts in installation packages, cannot rely on any of these interpreters being available in future, post-Catalina versions of macOS. Installer pkgs can be run in all kinds of environments and at all times, and you would not want them to fail, because a dependency is missing.

The good news is that we still have time. All the runtimes mentioned above are still present in Catalina, so the packages will continue to work for now. But if you are building installation scripts, you need to check if any of the installation scripts use one of these interpreters and fix them.

I recommend to use /bin/sh for installation scripts, since that will run in any macOS context, even the Recovery system.

If you are using third-party installer packages, you may also want to check them for these interpreters, and notify the developer that these packages will break in future versions of macOS.

To check a flat installer package, you would expand it with pkgutil --expand and then look at script files in the Scripts folder. This will work fine for a package or two, but gets tedious really quickly, especially with large distribution pkgs with many components (e.g. Office).

So… I wrote a script to do it. The script should handle normal component pkgs, distribution pkgs and the legacy bundle pkgs and mpkgs.

You can get the pkgcheck script from my Github repo.

What the script does

Once I had written the code to inspect all these types of pkgs, I realized I could grab all other kinds of information, as well. The pkgcheck.sh script will check for:

  • Signature and Notarization
  • Type of Package: Component, Distribution, legacy bundle or mpkg
  • Identifier and version (when present)
  • Install-location
  • for Distribution and mpkg types, shows the information for all components as well
  • for every script in a pkg or component, checks the first line of the script for shebangs of the deprecated interpreters (/bin/bash, /usr/bin/python, /usr/bin/perl, and /usr/bin/ruby) and print a warning when found

How to run pkgcheck.sh

Run the script with the target pkg file as an argument:

% ./pkgcheck.sh sample.pkg

You can give more than one file:

% ./pkgcheck.sh file1.pkg file2.pkg ...

When you pass a directory, pkgcheck.sh will recursively search for all files or bundle directories with the pkg or mpkg extension in that directory:

% ./pkgcheck.sh SamplePkgs

Features and Errors

There are a few more things that I think might be useful to check in this script. Most of all, I want to add an indicator whether a component is enabled by default or not. If you can think of any other valuable data to display, let me know. (Issue or Pull Request or just ping me on MacAdmins Slack)

I have tested the script against many pkgs that I came across. However, there are likely edge cases that I haven’t anticipated, which might break the script. If you run into any of those, let me know. (File an Issue or Pull Request.) Having the troublesome pkg would of course be a great help.

Note: the script will create a scratch directory for temporary file extractions. The script doesn’t actually expand the entire pkg file, only the Scripts sub-archive. The scratch folder will be cleaned out at the beginning of the next run, but not when the script ends, as you might want to do some further inspections.

Sample outputs

This is a sample pkg I build in my book, it has pre- and postinstall scripts using a /bin/bash shebang:

% ./pkgcheck.sh SourceCodePro-2.030d.pkg
SourceCodePro-2.030d
SamplePkgs/SourceCodePro-2.030d.pkg
Signature:      None
Notarized:      No
Type:           Flat Component PKG
Identifier:     com.example.SourceCodePro
Version:        2.030d
Location:       /
Contains 2 resource files
postinstall has shebang #!/bin/bash
preinstall has shebang #!/bin/bash

This is the experimental notarized pkg installer for desktoppr:

% ./pkgcheck.sh desktoppr-0.2.pkg
desktoppr-0.2
SamplePkgs/desktoppr-0.2.pkg
Signature:      Developer ID Installer: Armin Briegel (JME5BW3F3R)
Notarized:      Yes
Type:           Flat Component PKG
Identifier:     com.scriptingosx.desktoppr
Version:        0.2
Contains 0 resource files

And finally, this is a big one, the Microsoft Office installer: (they have some work to do to clean up those scripts)

% ./pkgcheck.sh Microsoft\ Office\ 16.27.19071500_Installer.pkg
Microsoft Office 16.27.19071500_Installer
SamplePkgs/Microsoft Office 16.27.19071500_Installer.pkg
Signature:      Developer ID Installer: Microsoft Corporation (UBF8T346G9)
Notarized:      No
Type:           Flat Distribution PKG
Contains 11 component pkgs

    Microsoft_Word_Internal
    Type:           Flat Component PKG
    Identifier:     com.microsoft.package.Microsoft_Word.app
    Version:        16.27.19071500
    Location:       /Applications
    Contains 3 resource files

    Microsoft_Excel_Internal
    Type:           Flat Component PKG
    Identifier:     com.microsoft.package.Microsoft_Excel.app
    Version:        16.27.19071500
    Location:       /Applications
    Contains 2 resource files

    Microsoft_PowerPoint_Internal
    Type:           Flat Component PKG
    Identifier:     com.microsoft.package.Microsoft_PowerPoint.app
    Version:        16.27.19071500
    Location:       /Applications
    Contains 2 resource files

    Microsoft_OneNote_Internal
    Type:           Flat Component PKG
    Identifier:     com.microsoft.package.Microsoft_OneNote.app
    Version:        16.27.19071500
    Location:       /Applications
    Contains 2 resource files

    Microsoft_Outlook_Internal
    Type:           Flat Component PKG
    Identifier:     com.microsoft.package.Microsoft_Outlook.app
    Version:        16.27.19071500
    Location:       /Applications
    Contains 2 resource files

    OneDrive
    Type:           Flat Component PKG
    Identifier:     com.microsoft.OneDrive
    Version:        19.70.410
    Location:       /Applications
    Contains 30 resource files
    postinstall has shebang #!/bin/bash
    od_logging has shebang #!/bin/bash
    od_service has shebang #!/bin/bash
    od_migration has shebang #!/bin/bash
    preinstall has shebang #!/bin/bash

    Office16_all_autoupdate
    Type:           Flat Component PKG
    Identifier:     com.microsoft.package.Microsoft_AutoUpdate.app
    Version:        4.13.19071500
    Location:       /Library/Application Support/Microsoft/MAU2.0
    Contains 2 resource files
    postinstall has shebang #!/bin/bash
    preinstall has shebang #!/bin/bash

    Office16_all_licensing
    Type:           Flat Component PKG
    Identifier:     com.microsoft.pkg.licensing
    Version:        16.27.19071500
    Location:       /
    Contains 2 resource files
    dockutil has shebang #!/usr/bin/python

    Office_fonts
    Type:           Flat Component PKG
    Identifier:     com.microsoft.package.DFonts
    Version:        0
    Location:       /private/tmp/com.microsoft.package.DFonts
    Contains 1 resource files
    postinstall has shebang #!/bin/bash

    Office_frameworks
    Type:           Flat Component PKG
    Identifier:     com.microsoft.package.Frameworks
    Version:        0
    Location:       /private/tmp/com.microsoft.package.Frameworks
    Contains 1 resource files
    postinstall has shebang #!/bin/bash

    Office_proofing
    Type:           Flat Component PKG
    Identifier:     com.microsoft.package.Proofing_Tools
    Version:        0
    Location:       /private/tmp/com.microsoft.package.Proofing_Tools
    Contains 1 resource files
    postinstall has shebang #!/bin/bash