Notarize a Command Line Tool with notarytool

When Apple introduced notarization with Catalina, I published a post describing how to notarize a command line tool. At WWDC this year, Apple introduced updates to this process with Xcode 13 (currently in beta). Most importantly, there is a new command line tool called notarytool.

While the previous, altool-based, workflow still works in Xcode 13, there are many advantages to the new notarytool which makes its use much simpler.

Apple has documented this tool in a WWDC21 session and some developer articles, in addition we got some great information through the twitter account of one of the engineers, and Howard Oakley has already written a post as well:

Update 2023-08-28: If you prefer to use Swift Package Manager to build command line tools, you can find instructions to package and notarize an SPM executable in this post.

What you need

  • Apple Developer Account (Personal or Enterprise, the free account does not provide the right certificates, nor access to the Xcode beta)
  • Xcode 13 (currently available as beta from the Apple Developer portal)
  • Developer ID Certificates
  • Application Specific Password for your Developer account
  • A command line tool project in Xcode

When you are building tools for macOS, you should have most of these already. We already covered these in the previous post, but to keep things in one place, I will cover them again, here.

Apple Developer Account

You need either the paid membership in the Apple Developer Program or be invited to an Apple Developer Enterprise Program team with access to the proper certificates.

You cannot get the required certificates with a free Apple Developer account, unless you are member of a team that provides access.

Xcode 13 (beta)

Until the full version of Xcode 13 is released, you can get Xcode 13 beta from the beta downloads page on the Apple Developer Portal.

Once it is released (usually when iOS is released) you will be able to download it from the Mac App Store, as well.

Xcode 13 requires macOS Big Sur 11.3 or higher. According to this tweet from Rosyna Keller, notarytool can be extracted and run on macOS Catalina 10.15.7 and higher.

You can run the notarytool binary through xcrun:

% xcrun notarytool --help

If you need to extract the binary you can find where is stored on disk with:

% xcrun --find notarytool
/Applications/Xcode-beta.app/Contents/Developer/usr/bin/notarytool

Developer ID Certificates

There are multiple certificates you can get from the Developer Program. By default you get a ‘Mac Developer’ certificate, which you can use for building and testing your own app locally.

To distribute binaries (apps and command line tools) outside of the App Store, you need a ‘Developer ID Application’ certificate. To sign installer packages for distribution outside of the Mac App Store, you need a ‘Developer ID Installer’ certificate.

We will need both types of Developer ID certificates, the first to sign the command line tool and the second to sign and notarize the installer package.

If you have not created these yet, you can do so in Xcode or in the Developer Portal. If you already have the certificates but on a different Mac, you need to export them and re-import them on the new Mac. Creating new certificates might invalidate the existing certificates! So beware.

Once you have created or imported the certificates on your work machine, you can verify their presence in the Terminal with:

% security find-identity -p basic -v

This command will list all available certificates on this Mac. Check that you can see the ‘Developer ID Application’ and ‘Developer ID Installer’ certificates. If you are a member of multiple teams, you may see multiple certificates for each team.

You can later identify the certificates (or ‘identities’) by the long hex number or by the descriptive name, e.g. "Developer ID Installer: Armin Briegel (ABCD123456)"

The ten character code at the end of the name is your Developer Team ID. Make a note of it, we will need it later. If you are a member of multiple developer teams, you can have multiple Developer ID certificates and the team ID will help you distinguish them.

Application Specific Password for your Developer Account

Apple requires Developer Accounts to be protected with two-factor authentication. To allow automated workflows which require authentication, you can create application specific passwords.

Note: If you followed the previous post’s instructions to store an application specific password for altool in the Keychain, you can extract that and re-use it for notarytool or create a new app-specific password.

Create a new application specific password in Apple ID portal for your developer account. Give it a name including notarytool so you know what you are using this for.

You will only be shown the password when you create it.

You can use notarytool to store the credentials in a keychain item, in a format that notarytool can read later.

% xcrun notarytool store-credentials --apple-id "name@example.com" --team-id "ABCD123456"

This process stores your credentials securely in the Keychain. You reference these credentials later using a profile name.

Profile name:
notary-example.com
Password for name@example.com: 
Validating your credentials...
Success. Credentials validated.
Credentials saved to Keychain.
To use them, specify `--keychain-profile "notary-example.com"`

The --store-credentials option will prompt for a profile name. You will need this name to retrieve the information later. Then it interactively prompts for the password associated with the given Apple Developer ID. Enter the application specific password here.

The credentials will be stored in the Keychain in an item named com.apple.gke.notary.tool. But you don’t really have to worry about that since notarytool will retrieve the credentials when you add the --keychain-profile "notary-example.com" option. (You can abbreviate the --keychain-profile with -p.)

If you are using iCloud Keychain, the credentials will be stored there, so they will be available to all other Macs you are using iCloud Keychain with. If you prefer, you can store the credentials in a specific (non-iCloud) keychain file with the --keychain option.

The Team ID is usually the 10-digit code which is also the certificates. However, in some cases the Team ID is different. You can can look-up Team IDs in the “Membership” area of the developer portal or with this altool command:

% xcrun altool --list-providers -u "name@example.com" -p "@keychain:<ITEM_NAME>"

(Thanks to ‘mhp’ for sharing this.)

You can also use an App Store Connect API key as an authentication option with notarytool. You can read notarytool‘s man page for details.

A Command Line Tool Project

You may already have a project to create a command line in Xcode. If you don’t have one, or just want a new one to experiment, you can just create a new project in Xcode and choose the ‘Command Line Tool’ template from ‘macOS’ section in the picker. The template creates a simple “Hello, world” tool, which you can use to test the notarization process.

My sample project for this article will be named “hello.”

Preparing the Xcode Project

The default settings in the ‘Command Line Tool’ project are suitable for building and testing the tool on your Mac, but need some changes to create a distributable tool.

The preparation in Xcode 13 diverges significantly from the steps required in the previous post. If you have created the project in earlier versions of Xcode, more configuration may be necessary.

Choosing the proper signing certificates

Before you can notarize the command line tool, it needs to be signed with the correct certificates.

  1. in Xcode, select the blue project icon in the left sidebar
  2. select the black “terminal” icon with your project’s name under the “Targets” list entry
  3. make sure the ‘Signing & Certificates’ tab is selected
  4. under ‘Signing’ disable ‘Automatically manage signing’
  5. choose your Team
  6. enter a bundle identifier for the binary
  7. choose ‘Developer ID Application‘ as the Signing Certificate

Hardened Runtime

Having the “Hardened Runtime” enabled is a requirement for notarization. When you create a new project in Xcode 13, the hardened runtime will be enabled by default. When you see the “Hardened Runtime” section under the “Signing” section, it is enabled.

When you are working with a older project, and do not see the “Hardened Runtime” section, you can enable the hardened runtime by clicking on the “+Capability” button above the “Signing” section and selecting “Hardened Runtime”.

Archive and export the binary

Choose “Archive” from the “Product” menu to build and create an archive. It will appear in the “Organizer” window. When that window does not open automatically, you can access it from the “Window” menu.

To export the binary product, select the latest archive and click on the “Distribute Content” button on the right. Choose “Built Products” as the method of distribution. Click “Next.” Choose a location to save the build products to.

This will create a directory with the project name and a timestamp in the chosen location. When you look inside this directory, you will see a “Products” directory and within it the binary in a /usr/local/bin/ directory hierarchy.

/usr/local/bin is the default location for command line tools in the Command Line Tool project template. It suits me fine most of the time, but you can change it by modifying the ‘Installation Directory’ build setting in Xcode and re-building the archive.

Build the installer package

Command Line Tools can be signed, but not directly notarized. You can however notarize a pkg file containing the Command Line Tool. Also, it is much easier for users and administrators to install your tool when it comes in a proper installation package.

We can use the Products directory as our payload to build the installer package:

% pkgbuild --root "hello 2021-mm-dd hh-mm-ss/Products" \
           --identifier "com.example.hello" \
           --version "1.0" \
           --install-location "/" \
           --sign "Developer ID Installer: Name (ABCD123456)" \
           hello-1.0.pkg

I have broken the command into multiple lines for clarity, you can enter the command in one line without the end-of-line backslashes \. You want to replace the values for the identifier, version and signing certificate with your data.

This will build an installer package which would install your binary on the target system. You should inspect the pkg file with Pacifist or Suspicious Package and do a test install on a test system to verify everything works.

If you want to learn more about installer packages and pkgbuild read my book “Packaging for Apple Administrators.”

Notarizing the Installer Package

Now we get to the new, most interesting part. We will notarize the newly-created installer package with notarytool:

% xcrun notarytool submit hello-1.0.pkg \
                   --keychain-profile "notary-scriptingosx" \
                   --wait

This is amazingly less effort than what we needed to do previously with the altool command. We give the filename of the archive we want to submit, the keychain profile with our credentials, and the --wait option.

notarytool will upload the file, give us a submission id, and then wait for the returned status from the Notary service. You can follow the output for the details.

You will also notice that notarytool uploads the pkg much faster than the previous altool workflow.

You can also drop the --wait option. Then the tool will submit the file and exit without waiting for a response. You can then use the info or log verbs with the submission id to get the status later. The Notary service does not seem to send emails anymore when the notarization check is complete.

There is also a --webhook option mentioned in the WWDC session which will make the Notary service call back to a webhook when the notarization is done. I have not seen any documentation on the details of this, though.

Finishing touch: stapler

Before you distribute the pkg, you can and should ‘staple’ the notarization before distributing it. This extra step will download the notarization information from Apple’s servers and attach it to the pkg. This is not mandatory, but will save the Gatekeeper service on the client an extra step when it verifies the pkg.

To do this, use the eponymous stapler tool:

% xcrun stapler staple hello-1.0.pkg

You can then verify that everything works with spctl:

% spctl --assess -vv --type install hello-1.0.pkg

Automation with Xcode

These steps are much simplified compared to the previous workflow. If you only build for distribution occasionally it would not be a big burden to do these steps manually.

Nevertheless, automating these steps saves effort and removes much pontential for errors.

When I wrote the previous post, I had not been able to figure out how all the pieces could work together to automate with a Xcode ‘Run Script’ as part of the normal “Archive” process. With the new tool and some inspiration from this developer article I have gotten this to work now.

In the project’s build settings, search for “Marketing Version” and set it to the version you want to use. Remember to update this entry for future updates as well. (You can use agvtool for this, but that is a topic for a different post.)

In Xcode, choose “Edit Scheme…” from the “Scheme” submenu in the “Project” menu. In the pane that opens, make sure the commnad line tool binary is selected at the top. Then expand the “Archive” section in the list on the left and select “Post-actions” in the expanded area. Use the “+” button at the bottom of the area to add a “New Run Script Action.”

Select the binary (again) in the popup next to “Provide build settings from”. Then paste the following in the code field:

With this post-action script in place, every “Archive” action will then also create a pkg in the project folder, submit it for notarization and staple the pkg. Since Xcode doesn’t show the output of post-action scripts, the script logs its output to a notary.log file, also in the project folder. Check that for success or failures. The notarization step takes a while after the “Archive” is complete, so you may have to wait a bit.

If you don’t want to run this workflow on every Archive, you can create a new scheme with this post-action script, then you can choose the scheme, before you do the “Archive” action.

Conclusion

The new notarytool included with Xcode 13 (beta) is a huge step up from the previous altool based workflows. It is much simpler and faster. You should start testing the tool now and move your workflows when possible.

Published by

ab

Mac Admin, Consultant, and Author