There is an updated version of this post for the new tools in Xcode 13.
The upcoming macOS 10.15 Catalina will require more apps and tools to be notarized. Apple has somewhat loosened the requirements at last minute, but these changed limitations are only temporary, to give developers more time to adapt.
Notarizing Mac Application bundles has its pitfalls, but is overall fairly well documented. However, I have been working on some command line tools written in Swift 5 and figured out how to get those properly signed and notarized.
Howard Oakley has written up his experiences and that post was extremely helpful. But there were a few omissions and some steps that aren’t really necessary, so I decided to make my own write-up.
And yes, there is a script at the end…
Note: these instructions are for macOS 10.14.6 Mojave, Xcode 10.3 and Swift 5.0. It is very likely that the details will change over time.
Update 2019-09-24: Tested with Xcode 11 and it still works (the screen layout has changed for some of the options)
What do you need?
- Apple Developer Account (Personal or Enterprise, the free account does not provide the right certificates)
- Xcode 10.3 or 11
- Developer ID Certificates (Application and Install)
- Application Specific Password for your Developer account
- a Command Line Tool Project that you want to sign and notarize
That’s a longish list. If you are already building command line tools in Xcode, you should have most of these covered already. We will walk through the list step-by-step:
Apple Developer Program Account
You need either the paid membership in the Apple Developer Program or be invited to an Apple Developer Enterprise Program team with access to the proper certificates.
You cannot get the required certificates with a free Apple Developer account, unless you are member of a team that provides access.
You can download Xcode from the Mac App Store or the developer download page. When you launch Xcode for the first time, it will prompt for some extra installations. Those are necessary for everything to in the article to work.
Developer ID Certificates
There are multiple certificates you can get from the Developer Program. By default you get a ‘Mac Developer’ certificate, which you can use for building and testing your own app locally.
To distribute binaries (apps and command line tools) outside of the App Store, you need a ‘Developer ID Application’ certificate. To sign installer packages for distribution outside of the Mac App Store, you need a ‘Developer ID Installer’ certificate.
We will need both types of Developer ID certificates, the first to sign the command line tool and the second to sign and notarize the installer package.
If you have not created these yet, you can do so in Xcode or in the Developer Portal. If you already have the certificates but on a different Mac, you need to export them and re-import them on the new Mac. Creating new certificates might invalidate the existing certificates! So beware.
Once you have created or imported the certificates on your work machine, you can verify their presence in the Terminal with:
% security find-identity -p basic -v
This command will list all available certificates on this Mac. Check that you can see the ‘Developer ID Application’ and ‘Developer ID Installer’ certificates. If you are a member of multiple teams, you may see multiple certificates for each team.
You can later identify the certificates (or ‘identities’) by the long hex number or by the descriptive name, e.g.
"Developer ID Installer: Armin Briegel (ABCD123456)"
The ten character code at the end of the name is your Developer Team ID. Make a note of it. If you are a member of multiple developer teams, you can have multiple Developer ID certificates and the team ID will help you distinguish them.
Application Specific Password for your Developer Account
Apple requires Developer Accounts to be protected with two-factor authentication. To allow automated workflows which require authentication, you can create application specific passwords.
Create a new application specific password in Apple ID portal for your developer account.
- Apple Support: Using app-specific passwords
You will only be shown the password when you create it. Immediately create a ‘New Password Item’ in your Keychain with the following fields:
- Keychain Item Name:
- Account Name: your developer account email
- Password: the application-specific password you just created
This will create a developer specific password item that we can access safely from the tools.
If you want, you can also store the app specific password in a different password manager, but the Xcode tools have a special option to use Keychain.
A Command Line Tool Project
You may already have a project to create a command line in Xcode. If you don’t have one, or just want a new one to experiment, you can just create a new project in Xcode and choose the ‘Command Line Tool’ template from ‘macOS’ section in the picker. The template creates a simple “Hello, world” tool, which you can use to test the notarization process.
My sample project for this article will be named “hello.”
Preparing the Xcode Project
The default settings in the ‘Command Line Tool’ project are suitable for building and testing the tool on your Mac, but need some changes to create a distributable tool.
Choosing the proper signing certificates
Before you can notarize the command line tool, it needs to be signed with the correct certificates.
- in Xcode, select the blue project icon in the left sidebar
- select the black “terminal” icon with your project’s name under the “Targets” list entry
- make sure the ‘General’ tab is selected
- under ‘Signing’ disable ‘Automatically manage signing’
- under ‘Signing (Debug)’ choose your Team and choose ‘Developer ID Application’ as the certificate
- under ‘Signing (Release)’ choose your Team and choose ‘Developer ID Application’ as the certificate
Enable Hardened Runtime
Enabling the ‘Hardened Runtime’ will compile the binary in a way that makes it harder for external process to inject code. This will be requirement for successful notarization starting January 2020.
- from the view where you changed the signing options, click on ‘Build Settings’ in the upper tab row
- click on ‘All’ to show all available settings
- enter ‘enable hardened’ in the search field, this will show the ‘Enable Hardened Runtime’ setting
- set the value in the project column (blue icon) to
Change the Install Build Location
If we want to automate the packaging and notarization, we need to know where Xcode builds the binary. The default location is in some
/tmp subdirectory and not very convenient. We will change the location for the final binary (the ‘product’) to the
build subdirectory in the project folder:
- in the same view as above, enter ‘Installation Build’ in the search field, this will show the ‘Installation Build Products Location’ setting
- double click on the value in the Project column (blue icon), this will open a popup window
- change the value to
If you manage your code in git or another VCS, you want to add the
build subdirectory to the ignored locations (
Build the Binary
You can use Xcode to write, test, and command line tool debug your. When you are ready to build and notarize a pkg installer, do the following:
- open Terminal and change directory to the project folder
% xcodebuild clean install
This will spew a lot of information out to the command line. You will see a
build subdirectory appear in the project folder, which will be filled with some directories with intermediate data.
After a successful build you should see a
pkgroot directory in the build folder, which contains your binary in the
/usr/local/bin is the default location for command line tools in the Command Line Tool project template. It suits me fine most of the time, but you can change it by modifying the ‘Installation Directory’ build setting in Xcode and re-building from the command line.
Build the pkg
Command Line Tools can be signed, but not directly notarized. You can however notarize a zip, dmg, or pkg file containing a Command Line Tool. Also, it is much easier for users and administrators to install your tool when it comes in a proper installation package.
We can use the
pkgroot directory as our payload to build the installer package:
% pkgbuild --root build/pkgroot \ --identifier "com.example.hello" \ --version "1.0" \ --install-location "/" \ --sign "Developer ID Installer: Armin Briegel (ABCD123456)" \ build/hello-1.0.pkg
I have broken the command into multiple lines for clarity, you can enter the command in one line without the end-of-line backslashes
\. You want to replace the values for the identifier, version and signing certificate with your data.
This will build an installer package which would install your binary on the target system. You should inspect the pkg file with Pacifist or Suspicious Package and do a test install on a test system to verify everything works.
If you want to learn more about installer packages and
pkgbuildread my book “Packaging for Apple Administrators.”
Notarizing the Installer Package
Xcode has a command line tool
altool which you can use to upload your tool for notarization:
xcrun altool --notarize-app \ --primary-bundle-id "com.example.com" \ --username "email@example.com" \ --password "@keychain:Developer-altool" \ --asc-provider "ABCD123456" \ --file "build/hello-1.0.pkg"
username is your developer account email.
asc-provider is your ten digit Team ID. If you are only a member in a single team you do not need to provide this.
The password uses a special
@keychain: keyword that tells
altool to get the app-specific password out of a keychain item named
Developer-altool. (Remember we created that earlier?)
This will take a while. When the command has successfully uploaded the pkg to Apple’s Notarization Servers, it will return a
RequestUUID. Your notarization request will be queued and eventually processed. You can check the status of your request with:
xcrun altool --notarization-info "Your-Request-UUID" \ --username "firstname.lastname@example.org" \ --password "@keychain:Developer-altool"
Apple will also send an email to your developer account when the process is complete. I my experience this rarely takes more than a minute or two. (Being in Central EU time zone might be an advantage there). When the process is complete, you can run the above
notarization-info command to get some details. The info will include a link that contains even more information, which can be useful when your request is rejected.
Note that the info links expire after 24 hours or so. You should copy down any information you want to keep longer.
Completing the Process
You will not receive anything back from Apple other than the confirmation or rejection of your request. When a Mac downloads your installer package and verifies its notarization status it will reach out to Apple’s Notarization servers and they will confirm or reject the status.
If the Mac is offline at this time, or behind a proxy or firewall that blocks access to the Apple Servers, then it cannot verify whether your pkg file is notarized.
You can, however, ‘staple’ the notarization ticket to the pkg file, so the clients do not need to connect to the servers:
% xcrun stapler staple build/hello-1.0.pkg
You can also use
stapler to verify the process went well:
% xcrun stapler validate build/hello-1.0.pkg
stapler depends on the developer tools to be installed, you should generally prefer
spctl to check notarization:
% spctl --assess -vvv --type install build/hello-1.0.pkg
Automating the Process
Obviously, I built a script to automate all this. Put the following script in the root of the project folder, modify the variables at the start of the script (lines 20–38) with your information, and run it.
The script will build the tool, create a signed pkg, upload it for notarization, wait for the result, and then staple the pkg.
You can use this script as an external build tool target in Xcode. There are other ways to integrate scripts for automation in Xcode, but all of this is a new area for me and I am unsure which option is the best, and which I should recommend.
Links and Videos
These links and videos, especially Howard Oakley’s post and Tom Bridge’s PSU Presentation have been hugely helpful. Also thanks to co-worker Arnold for showing me this was even possible.
- Howard Oakley: Building and delivering command tools for Catalina
- Tom Bridge: PSU Conference Talk and Followup
- Apple Support: Notarizing Your Mac Software for macOS Catalina
- Apple Support: Using app-specific passwords
- Apple Developer: Notarizing Your App Before Distribution
- WWDC 2019: All About Notarization
- WWDC 2019: Advances in macOS Security
- Apple Developer: Resolving Common Notarization Issues
Notarization is a key part of Apple’s security strategy going in macOS.
As MacAdmins we will usually deploy software through management systems, where the Gatekeeper mechanisms which evaluate notarization are bypassed. There are, however, already special cases (Kernel Extensions) where notarization is mandatory. It is likely that Apple will continue to tighten these requirements in the future. The macOS Mojave 10.14.5 update has shown that Apple may not even wait for major releases to increase the requirements.
If you are building your own tools and software for macOS and plan to distribute the software to other computers, you should start signing and notarizing.
On the other hand, I find the introduction of Notarization to macOS encouraging. If Apple wanted to turn macOS into a “App Store only system” like iOS, they would not have needed to build the notarization process and infrastructure. Instead, Apple seems to have embraced third-party-software from outside the App Store.
Notarization allows Apple to provide a security mechanism for software distributed through other means. It cannot be 100% effective, but when used correctly by Apple and the software developers it will provide a level of validation and trust for software downloaded from the internet.
7 thoughts on “Notarize a Command Line Tool”
Awesome article, thanks for sharing!
I’m adapting your pkgAndNotarize.sh to our needs so it can be ran from the CI, and thought it would be useful to output the actual error from xcrun should it fail.
Here’s what was done: https://pastebin.com/B7iW0Ew1 . Basically I broke down the command that was assigned to $requestUUID into two parts, one containing the output from xcrun and another doing the awk parsing. This way it’s possible to show the (error) output from xcrun if the $requestUUID ends up empty!
Your article hands down is the best Notary process guide for developers.
Appreciate the effort.
I now understand the Notary process thanks to you and
I will try your script and revert too.
I appreciate that all the steps are in one place. Thanks for putting this together.
For some the `asc-provider` is not the team id, so `dev-team` in the script should be the `ProviderShortname` from `xcrun altool –list-providers -u “AC_USERNAME” -p “@keychain:AC_PASSWORD”`
Can you elaborate as to how you can notarize a package created with a tool like pyinstaller?
I have have a mess of python that I need to deploy and I am struggling to figure out how to do this.
No, since I only have a superficial knowledge of python and none of pyinstaller. I really can’t.
Thanks for the replay!
I’ve managed to get to a point where I can build a command line tool and sign it. I have even successfully managed to have a .dmg and .zip notarized, but inevitably when I deploy those the hosts always raise the “unidentified developer” messages.
Any idea where I should start diagnosing this?
Comments are closed.