Scripting macOS, part 2: The Script File

This series is an excerpt from the first chapter of my upcoming book “Scripting macOS” which will teach you to use and create shell scripts on macOS.

  • Part 1: First Script
  • Part 2: The Script File
  • Part 3: The Code
  • Part 4: Running the Script
  • Part 5: Lists of Commands
  • Part 6: Turning it off and on again

I will publish one part every week over the summer. Enjoy!

The Script File

In the previous part, we created a text file and named it hello.sh. As we have mentioned before, all shell script files are text files.

File Extension

The file extension .sh is entirely optional. When you rename the file to merely hello it will remain a functional script:

> mv hello.sh hello
> ./hello
Hello, World!

When you use the script frequently as a command line tool, typing the additional .sh is cumbersome and does not ‘fit in’ very well with all the other command line tools. It is very common to remove the file extension in these cases.

However, the .sh file extension tells Finder and other applications what kind of file this is. You can assign the .sh file extension to be opened in your favored text editor when you double-click the file in Finder.

To do this, rename the file so it has the extension again:

> mv hello hello.sh

Then select the script file in Finder and open its Info panel (⌘I).

In the ‘General’ section of the info panel, you can see that Finder recognizes this file extension as a shell script.

In the ‘Open with’ section you can tell Finder which application to use to open this file. The popup list will show all applications you have installed that can open .sh files. Select your favorite text editor.

To tell the system to open all files with a .sh extension with your favorite text editor, click on the ‘Change All…’ button here. This will set your favorite text editor as the default application for the .sh file extension.

Some people like to use .bash and .zsh file extensions to further distinguish scripts written specifically in bash or zsh. While this can be useful in some workflows—especially when translating scripts from bash to zsh—it is generally not necessary. The .sh file extension works well for all these scripts. The .bash and .zsh file extensions are also not automatically recognized by the Finder or text editors as shell scripts.

Executable Bit

Aside from the actual script code, a shell script requires one more thing that distinguishes it from any other text file. You have to the tell the system that this file can be executed as a command.

This information is stored in the executable bit of the file privileges or file mode. You can see this information with the long form of the ls command:

> ls -l hello.sh 
-rwxr-xr-x@ 1 armin  staff  hello.sh

The file mode is displayed in the first ten characters in this output: -rwxr-xr-x.

The leading dash designates this item as a file. (A directory will show d, a symbolic link will show l, etc.)

The file access privileges are shown next. There are three sets of three flags or bits. The first three (in this case rwx) are the privileges for the file’s owner. The second set (r-x) are the privileges for group access and the last set (also r-x) shows the privileges for all other user on this system.

The three flags in each set are r, w, and x, in that order. They stand for read (r), write (w) and execute (x) and show the actions the users associated with each set can perform.

The file’s owner can read, write (or change), and execute (or run) this file. Users who are a member of the staff group, can read and execute this file, as well as every other user on this system.

Most documents and files merely contain text and data. The execute bit is not required. When you create a new file, the executable flag is disabled by default:

> touch newfile.txt
> ls -l newfile.txt
-rw-r--r--  1 armin  staff  newfile.txt

Having the execute bit disabled by default is a sensible security measure. You will have to explicitly enable it when building scripts or other executables.

When you are writing shell scripts, you eventually want to execute them. To do this you have to enable the execute bit. When then execute bit is unset, you cannot run the script from the command line.

To demonstrate this—and to gain some familiarity with the command line tools required to manipulate the file mode—disable the executable bit of our hello.sh script:

> chmod -x hello.sh

You can verify that the executable bit has been removed with the ls -l command:

> ls -l hello.sh
-rw-r--r--@ 1 armin  staff  45 Feb 17 14:36 hello.sh

When you now attempt to run the command, it will fail and the shell will tell you the reason: you do not have the right permissions. Without the executable script, your shell script is merely a text file.

> ./hello.sh
zsh: permission denied: ./hello.sh

You can re-set the executable bit with

> chmod +x hello.sh

You do not need write (w) permissions to execute a script, but you require read (r) permissions.

You can remove the executable privilege for group and/or other users for security. Since most Macs are used as single user systems this is not really relevant here. When you create, install, and run shell scripts on shared servers with other Unix or unix-like servers with many users, the proper access privileges may be extremely important. Please consult with the system administrator.

You will usually have to remember to set the executable bit when you create a new script. Some text editors may set it for you when they recognize you are working on a shell script.

Note: The executable bit has a different meaning for directories than it does for files. You can find the details in the chmod man page or in my book ‘macOS Terminal and Shell

Weekly News Summary for Admins — 2021-07-23

Apple realeased iOS 14.7, with watchOS and tvOS updates as well. Later in the week, on Wednesday we got macOS 11.5 and iPadOS 14.7. iOS 14.7 brings support for the new battery pack. For MacAdmins, macOS 11.5 finally delivers an option to lock booting to Recovery and a device lock for Apple Silicon Macs.


(Sponsor: SentinelOne)

Bypassing macOS TCC User Privacy Protections By Accident and Design

For Mac Admins, TCC presents a host of challenges. Keeping users and admins productive can unexpectedly lead to undermining these protections. Learn about TCC ‘gotchas’, known bypasses and how to stay safe.

Continue reading on SentinelLabs


Anthony Reimer wrote a greate piece called “Recognition, Retirement, and Remembrance” which resonated with many other admins.

When I started out the news letter, the goal was to feature the work that so many people in the Apple Administrator community publish on their weblogs, social media, YouTube, podcasts, and open source repositories. It is really hard to keep track of everything that is going on when you are not a news junkie.

I love when people ping me on Slack to point out an interesting post (please keep them coming). I love it even more when someone sends me their first post on a new blog or their first post in a long time. (Congratulations!)

This discussion has made me even more aware of this role. When I curate the list of links I gathered over the week, I have always tried to be more inclusive, especially on the blog posts.

I have been working most of my professional career in or at least adjacent to the MacAdmin community. I have no comparison how this works in other professional communities. But this willingness to communicate and share, this notion of being in this together feels special.

This news letter would not exist without all you people willing to share their knowledge, experiences, and, yes, frustrations. You take time out of your work and volunteer it to the community, so that all our work, lives, and experiences become just a bit easier.

Thank you!


This news letter is going on vacation break! The next one will be sent out on August 27, 2021, with all the things that happened in the mean time.

So you still have something to read, I have scheduled a series of posts to be published on my website every week until I get back. The series is an excerpt from the first chapter of my upcoming book “Scripting macOS.” The first part is up already, I hope you like it!


If you would rather get the weekly newsletter by email, you can subscribe to the Scripting OS X Weekly Newsletter here!! (Same content, delivered to your Inbox once a week.)

macOS and iOS Updates

Note: I am linking to the US support pages, as the updates may not have been published to all localized pages yet.

Reactions

MacAdmins on Twitter

  • Sören: “Want old-style proxy icons and a bigger grabbable title bar back? defaults write http://com.apple.finder NSWindowSupportsAutomaticInlineTitle -bool false and killall Finder!”
  • mikeymikey: “I don’t know who needs to hear this, but shift option double click on the corner of a window makes it fill the whole screen top to bottom and left to right without going fullscreen app (or triggering “zoom”, which never does what I want).”
  • Anthony Reimer: “You don’t need Shift from my experiments today. Double-click any window edge: window expands in that direction as far as possible. Double-click any window corner: window expands in 2 directions. Add Option key when double-clicking: opposite side(s) of window also expands.”

Security and Privacy

Support and HowTos

Updates and Releases

To Watch

To Listen

Just for Fun

Support

If you are enjoying what you are reading here, please spread the word and recommend it to another Mac Admin!

If you want to support me and this website even further, then consider buying one (or all) of my books. It’s like a subscription fee, but you also get a useful book or two extra!

Scripting macOS, part 1: First Script

This series is an excerpt from the first chapter of my upcoming book “Scripting macOS” which will teach you to use and create shell scripts on macOS.

  • Part 1: First Script
  • Part 2: The Script File
  • Part 3: The Code
  • Part 4: Running the Script
  • Part 5: Lists of Commands
  • Part 6: Turning it off and on again

I will publish one part every week over the summer. Enjoy!

First Script

When learning programming languages it is tradition that the first program you write displays ‘Hello, World!’

Printing these words to the screen is usually a really simple task. In most programming languages it requires just a single line. Nevertheless, creating a simple program or script like this will teach you a lot about all the other tasks you need to create and run a script.

Before you start, you should create a folder where you store all the scripts you will create. A Documents subfolder is a good location. I have a Projects folder in my home directory where I create subfolders for all my scripting and programming projects. You may already have some structure like this that makes sense for you.

You can create this in Finder and the navigate there in Terminal. But, just to practice working in Terminal, you can create your script project folder in the command line:

> cd ~/Documents
> mkdir ScriptingMacOS
> cd ScriptingMacOS

Where- and however you create this folder, we will refer to it as your script project folder from now on. When you interact with the scripts from the command line, always remember to change the working directory there.

Note: When you know how to use a version control system to manage code, such as git, you can set up or initialize this project folder as repository now. Shell scripts are well suited to be managed with version control systems and I would recommend to use it when building scripts. But explaining the details of git as well as teaching shell scripting would be overwhelming and beyond the scope of this series.

Create your first script

Create a new text file in your favored text editor.

Note: If you do not have a favorite text editor yet, I recommend BBEdit. You can use it for free or pay the license fee to unlock the full feature set. After installing BBEdit, be sure to run ‘Install Command Line Tools…’ from the BBEdit menu.

I will be using the bbedit command line tools in my examples, but other text editors have similar commands.

When you are using a text editor with a command line tool, you can create a new empty file from the command line and open it in the text editor like this:

> bbedit hello.sh

Enter the following text into the text document:

#!/bin/zsh

# Greetings
echo "Hello, World!"

Tip: You can copy and paste the code from this post into the text editor. In general, I approve of this, since it speeds up the process and avoids typing errors. But in the beginning, I would recommend typing out the commands and modifying the script manually. This will engage your brain and memory differently and help you memorize some of the standard steps and commands.

Save the text file you created to a file named hello.sh in the script project folder.

In Terminal, ensure that the working directory is your script project folder. Then enter the following:

> chmod +x hello.sh

This will make your script file executable. You are telling the system this file has code that can be run as a program rather than just data. We will look at what this means in detail later.

Now you can run or execute your script:

> ./hello.sh
Hello, World!

If you get a different result, please verify that you typed the script exactly as given above. Pay extra attention to spaces, quotes and other special characters. Then save the text file and try running the script again:

> ./hello.sh
Hello, World!

Even for a script as simple as this, there is a lot that happened here. We will look at the pieces and steps in detail in the next part.

Suspicious Package Power User Features

As many MacAdmins, I work a lot with installer packages. You can say I wrote the book about it. When you get an installer package from some vendor website, you will want to inspect it before you install it anywhere, let alone deploy it to dozens, hundreds, or even thousands of Macs in your fleet.

You can use the pkgutil tool to do this in the command line, but there are package inspector tools with a graphical interface that are very useful and popular.

One of these tools is Suspicious Package from Mothers Ruin Software. It displays all of the content and resources of a pkg file in a very nice user interface. Many people love Suspicious Package from Mothers Ruin Software. I have always had reservations about Suspicious Package, though, because I thought there were a few missing features.

The missing features were in connection with distinguishing “normal” or “component” packages (which have a single payload) from “distribution” or “meta” packages (which don’t have a payload of their own, but contain one or more component packages).

I have explained the differences in a bit more detail in my MacDevOps YVR presentation “The Encyclopedia of Packages” where I (once again, ignorantly) stated that you can’t really tell them apart in the UI of Suspicious Package.

After that presentation, Mat X, one of the organizers of the conference got me in touch with the developer of Suspicious Package, Randy Saldinger, who graciously and patiently demonstrated that I was wrong.

In my defense, you really cannot tell normal packages from distribution packages in the default configuration of Suspicious Package, but if I had bothered to read the manual and/or explore the Preferences window, I would have found this option:

This will show the Distribution xml file at the top of the list of the ‘All Scripts’ pane for distribution packages. When you see no Distribution file there, the package is a component package.

The second checkmark in that preference window is also very useful. With “Component package and bundle info” enabled you can see which component contains the selected file in the info pane:

You can also search in the “All Files” tab with command-F and use the component package ID as a search criteria.

All of this is already well enough to remove the reservations I have had on Suspicious Package. But Randy shared another preference with me which puts it over the top. It is not exposed in the UI (yet) but when you run:

% defaults write com.mothersruin.SuspiciousPackage ShowRawPackageInfo -bool YES

in Terminal and re-launch Suspicious Package, you will see the raw PackageInfo xml in the “All Scripts” tab. Together with showing the Distribution xml, this allows you to inspect all the raw metadata that can be inside a pkg file.

I have also learned that you can use the search functionality in the “Help” menu of Suspicious Package which will link directly to the online documentation. Not many apps leverage this functionality, so we often forget to check for it. Kudos to Suspicious Package for using this.

Many thanks to Randy for all his work and help and for providing an excellent tool! I am very much looking forward to the next version.

Weekly News Summary for Admins — 2021-07-16

In a break from our usual blogging, Anthony Reimer and I did a product review. He will explain the background and the tracking part, and I reviewed the receiving end.


(Sponsor: Mosyle)

Mosyle Fuse logo

The Fusion of Apple MDM, Identity, Patching & Security.

Mosyle Fuse is the first and only product to bring a perfect blend of an Enterprise-grade MDM, an innovative solution for macOS Identity Management, automated application installation and patching, and purpose-built multi-layer endpoint security, all specially designed for Apple devices used at work at a price point that’s almost unexplainable.

Click here to learn more!


And we got macOS 11.5 and iOS 14.7 release candidates, as well as macOS 12 and iOS 15 beta 3. Keep testing and feeding back!

If you would rather get the weekly newsletter by email, you can subscribe to the Scripting OS X Weekly Newsletter here!! (Same content, delivered to your Inbox once a week.)

macOS 12 Monterey and iOS 15

Security and Privacy

Support and HowTos

Updates and Releases

To Listen

Just for Fun

Support

If you are enjoying what you are reading here, please spread the word and recommend it to another Mac Admin!

If you want to support me and this website even further, then consider buying one (or all) of my books. It’s like a subscription fee, but you also get a useful book or two extra!

Installomator v0.6

We have posted an update for Installomator, which brings it to v0.6.

The changes are as follows:

  • several new and updated labels, for a total of 302
  • versionKey variable can be used to choose which Info.plist key to get the version from
  • an appCustomVersion() {} function can now be used in a label
  • with INSTALL=force, the script will not be using updateTool, but will reinstall instead
  • added quit and quit_kill options to NOTIFY
  • updated buildCaseStatement.sh
  • updated buildInstallomatorPkg.sh to use notarytool (requires Xcode 13)
  • several minor fixes

There have been some other organizational changes as well. We have moved the repo to its own team on GitHub: Installomator/Installomator. This should reflect that I am no longer the sole, or even the main contributor. Many thanks to Søren Theilgaard, Isaac Ordonez, and Adam Codega for helping maintain this!

And many thanks to everyone else who contributed!

An AirTag Adventure, Part 2—Receiving an AirTag

Anthony Reimer and I had a lot of fun sending an AirTag across the Atlantic. Now we get to the experience of being on the receiving side.

“AirTag Found Moving With You”

The “Find My” network warns you when an unknown AirTag is moving with you. This is to prevent tracking people without their approval. Since Anthony had registered the AirTag on his account to track its travel, this was a perfect opportunity to test this situation.

I just dropped the AirTag in the front pocket of my backpack and went about my business. This is the backpack I use to transport odds and ends, especially groceries, so it nearly always goes where I go. At first the backpack remained immobile in my house for the afternoon and then overnight. The next morning, we took a trip to the Leiden Farmers’ Market when I got the warning that an AirTag was moving with me.

It is interesting (but makes sense) that the warning didn’t come until I actually started moving with the AirTag. The tag is not really tracking you when it is just sitting around. But once I was on the go (together with the “strange” AirTag) I was warned fairly quickly: after about 2.5 hours. My wife also got the same warning on her iPhone, which should not be surprising, since we were walking together.

The AirTag is supposed to eventually make a sound when it is separated from its owner, but it never got to that phase in our testing, or I did not hear it.

The iPhone showed me a map where my iPhone had detected the “strange” AirTag and offered the option to play a sound on the AirTag to help locate it. Presumably, when someone tracks you without your knowledge, the AirTag would be hidden somewhere nearby and the sound will help you find it.

You can tell your phone to ignore this particular AirTag, presumably after you have checked with your companions who are travelling with you or because you are carrying a borrowed, tagged item.

The app also shows the serial number of the AirTag and the last four digits of the phone number it is registered to. These digits should allow you to identify the owner, when you know them, but maintain their anonymity when you don’t.

You can also get instructions to disable the AirTag. This will show instructions to open it and remove the battery with a simple but effective animation. (AirTags open real easy, given how small they are and how solidly sealed they seem when closed. This is really impressive engineering.)

I can see a possible downside here. The disabling process requires you to actually find the AirTag. If someone manages to hide the AirTag in way that you cannot find or access it physically, you cannot disable it. This might be harder than I imagine, because shielding the AirTag in a way that muffles the sound sufficiently might also shield the Bluetooth transmissions, which prevent the tracking in the first place. More experimentation will be needed here.

Lost Mode

Now that the covert tracking had been tested, Anthony set the AirTag to lost mode on his account. It took a few minutes for that change to propagate through the network. With lost mode enabled, I could call up his contact information (the owner can choose whether to show an email or the phone number) from the AirTag on my phone, just by tapping my phone to the AirTag.

Anthony also tried to play a sound on the AirTag, which was more than 7000km away from him. This did not work. It seems that playing the sound requires a local bluetooth connection to the AirTag. Since you would likely not be able to hear the sound when you are out of bluetooth range, and could use this to ‘terrorize’ someone (intentionally or not) in the middle of the night, I think this a reasonable limitation.

Transferring the AirTag

With all our testing done it was time for Anthony to remove the AirTag from his account, so that I could add it to my account. The interface for that in the Find My application is very straightforward.

He did, however, get an error that the iPhone could not “find” the AirTag. We presume his iPhone tried to connect to the AirTag over local bluetooth to let it “know” it was removed.

After Anthony had removed the device from his account, I tried to set it up on mine. This did not immediately work. Even after waiting for a few hours, my phone would not recognize the AirTag as new.

I then followed the instructions in this support article to reset the AirTag. It’s a bit tedious as you have to remove and replace the battery five times in a row. I figured out you don’t have to actually close the lid five times, just taking out the battery and putting it back in its place is sufficient. (there are magnets in the AirTag that seem to hold the batttery in place) After the reset process, the AirTag appeared immediately for setup on my phone and I could add it to my iCloud account.

Conclusion

Overall, the user experience for both the “Moving with you” and “Lost Mode” workflows are well thought through and kept clear and simple. Apple has good support articles for reference.

Many thanks to the comittee of MacDeployment and their sponsors that provided AirTags to all the speakers. And thanks to Anthony, who was game when I suggested that sending and tracking an AirTag across the Atlantic would be the “most fun” way to get them to me. Hope you found our experiments interesting, as well!

Right now, the AirTag has returned to my backpack. This seems reasonable since it stores my wallet and keys when I leave the house. I also want to test attaching an AirTag to my bike. I believe that bike thieves will quickly catch on to AirTags, so I don’t have high hopes for it to be useful as theft prevention. But an AirTag on the bike should be very useful to find my bike again in one of the typical Dutch bike parking areas among thousands of other bikes.

Scripting OS X — Weekly News Summary for Admins — 2021-07-09

Summer doldrums are here! This is a good thing, because it gives us MacAdmins more time to test the upcoming betas. You are testing and providing feedback, right?

Since I mentioned betas, macOS 11.5 and iOS 14.7 beta5 came out this week, as well!


(Sponsor: Mosyle)

Mosyle Fuse logo

The Fusion of Apple MDM, Identity, Patching & Security.

Mosyle Fuse is the first and only product to bring a perfect blend of an Enterprise-grade MDM, an innovative solution for macOS Identity Management, automated application installation and patching, and purpose-built multi-layer endpoint security, all specially designed for Apple devices used at work at a price point that’s almost unexplainable.

Click here to learn more!


In other news, I got my second Corona vaccination shot today. Hooray for science and technology! You might enjoy imagining this newsletter getting loopier and loopier as I progress into immune reaction stupor… 😉

If you would rather get the weekly newsletter by email, you can subscribe to the Scripting OS X Weekly Newsletter here!! (Same content, delivered to your Inbox once a week.)

News and Opinion

macOS 12 Monterey and iOS 15

MacAdmins on Twitter

  • mikeymikey: “Take the amount of time it would take to manually install and configure critical apps and tools for an employee role and multiply it out by how much of their payroll you’d be wasting without IT ensuring it’s in self-service”

Security and Privacy

Support and HowTos

Scripting and Automation

Updates and Releases

To Listen

Just for Fun

Support

If you are enjoying what you are reading here, please spread the word and recommend it to another Mac Admin!

If you want to support me and this website even further, then consider buying one (or all) of my books. It’s like a subscription fee, but you also get a useful book or two extra!

Notarize a Command Line Tool with notarytool

When Apple introduced notarization with Catalina, I published a post describing how to notarize a command line tool. At WWDC this year, Apple introduced updates to this process with Xcode 13 (currently in beta). Most importantly, there is a new command line tool called notarytool.

While the previous, altool-based, workflow still works in Xcode 13, there are many advantages to the new notarytool which makes its use much simpler.

Apple has documented this tool in a WWDC21 session and some developer articles, in addition we got some great information through the twitter account of one of the engineers, and Howard Oakley has already written a post as well:

What you need

  • Apple Developer Account (Personal or Enterprise, the free account does not provide the right certificates, nor access to the Xcode beta)
  • Xcode 13 (currently available as beta from the Apple Developer portal)
  • Developer ID Certificates
  • Application Specific Password for your Developer account
  • A command line tool project in Xcode

When you are building tools for macOS, you should have most of these already. We already covered these in the previous post, but to keep things in one place, I will cover them again, here.

Apple Developer Account

You need either the paid membership in the Apple Developer Program or be invited to an Apple Developer Enterprise Program team with access to the proper certificates.

You cannot get the required certificates with a free Apple Developer account, unless you are member of a team that provides access.

Xcode 13 (beta)

Until the full version of Xcode 13 is released, you can get Xcode 13 beta from the beta downloads page on the Apple Developer Portal.

Once it is released (usually when iOS is released) you will be able to download it from the Mac App Store, as well.

Xcode 13 requires macOS Big Sur 11.3 or higher. According to this tweet from Rosyna Keller, notarytool can be extracted and run on macOS Catalina 10.15.7 and higher.

You can run the notarytool binary through xcrun:

% xcrun notarytool --help

If you need to extract the binary you can find where is stored on disk with:

% xcrun --find notarytool
/Applications/Xcode-beta.app/Contents/Developer/usr/bin/notarytool

Developer ID Certificates

There are multiple certificates you can get from the Developer Program. By default you get a ‘Mac Developer’ certificate, which you can use for building and testing your own app locally.

To distribute binaries (apps and command line tools) outside of the App Store, you need a ‘Developer ID Application’ certificate. To sign installer packages for distribution outside of the Mac App Store, you need a ‘Developer ID Installer’ certificate.

We will need both types of Developer ID certificates, the first to sign the command line tool and the second to sign and notarize the installer package.

If you have not created these yet, you can do so in Xcode or in the Developer Portal. If you already have the certificates but on a different Mac, you need to export them and re-import them on the new Mac. Creating new certificates might invalidate the existing certificates! So beware.

Once you have created or imported the certificates on your work machine, you can verify their presence in the Terminal with:

% security find-identity -p basic -v

This command will list all available certificates on this Mac. Check that you can see the ‘Developer ID Application’ and ‘Developer ID Installer’ certificates. If you are a member of multiple teams, you may see multiple certificates for each team.

You can later identify the certificates (or ‘identities’) by the long hex number or by the descriptive name, e.g. "Developer ID Installer: Armin Briegel (ABCD123456)"

The ten character code at the end of the name is your Developer Team ID. Make a note of it, we will need it later. If you are a member of multiple developer teams, you can have multiple Developer ID certificates and the team ID will help you distinguish them.

Application Specific Password for your Developer Account

Apple requires Developer Accounts to be protected with two-factor authentication. To allow automated workflows which require authentication, you can create application specific passwords.

Note: If you followed the previous post’s instructions to store an application specific password for altool in the Keychain, you can extract that and re-use it for notarytool or create a new app-specific password.

Create a new application specific password in Apple ID portal for your developer account. Give it a name including notarytool so you know what you are using this for.

You will only be shown the password when you create it.

You can use notarytool to store the credentials in a keychain item, in a format that notarytool can read later.

% xcrun notarytool store-credentials --apple-id "name@example.com" --team-id "ABCD123456"

This process stores your credentials securely in the Keychain. You reference these credentials later using a profile name.

Profile name:
notary-example.com
Password for name@example.com: 
Validating your credentials...
Success. Credentials validated.
Credentials saved to Keychain.
To use them, specify `--keychain-profile "notary-example.com"`

The --store-credentials option will prompt for a profile name. You will need this name to retrieve the information later. Then it interactively prompts for the password associated with the given Apple Developer ID. Enter the application specific password here.

The credentials will be stored in the Keychain in an item named com.apple.gke.notary.tool. But you don’t really have to worry about that since notarytool will retrieve the credentials when you add the --keychain-profile "notary-example.com" option. (You can abbreviate the --keychain-profile with -p.)

If you are using iCloud Keychain, the credentials will be stored there, so they will be available to all other Macs you are using iCloud Keychain with. If you prefer, you can store the credentials in a specific (non-iCloud) keychain file with the --keychain option.

The Team ID is usually the 10-digit code which is also the certificates. However, in some cases the Team ID is different. You can can look-up Team IDs in the “Membership” area of the developer portal or with this altool command:

% xcrun altool --list-providers -u "name@example.com" -p "@keychain:<ITEM_NAME>"

(Thanks to ‘mhp’ for sharing this.)

You can also use an App Store Connect API key as an authentication option with notarytool. You can read notarytool‘s man page for details.

A Command Line Tool Project

You may already have a project to create a command line in Xcode. If you don’t have one, or just want a new one to experiment, you can just create a new project in Xcode and choose the ‘Command Line Tool’ template from ‘macOS’ section in the picker. The template creates a simple “Hello, world” tool, which you can use to test the notarization process.

My sample project for this article will be named “hello.”

Preparing the Xcode Project

The default settings in the ‘Command Line Tool’ project are suitable for building and testing the tool on your Mac, but need some changes to create a distributable tool.

The preparation in Xcode 13 diverges significantly from the steps required in the previous post. If you have created the project in earlier versions of Xcode, more configuration may be necessary.

Choosing the proper signing certificates

Before you can notarize the command line tool, it needs to be signed with the correct certificates.

  1. in Xcode, select the blue project icon in the left sidebar
  2. select the black “terminal” icon with your project’s name under the “Targets” list entry
  3. make sure the ‘Signing & Certificates’ tab is selected
  4. under ‘Signing’ disable ‘Automatically manage signing’
  5. choose your Team
  6. enter a bundle identifier for the binary
  7. choose ‘Developer ID Application‘ as the Signing Certificate

Hardened Runtime

Having the “Hardened Runtime” enabled is a requirement for notarization. When you create a new project in Xcode 13, the hardened runtime will be enabled by default. When you see the “Hardened Runtime” section under the “Signing” section, it is enabled.

When you are working with a older project, and do not see the “Hardened Runtime” section, you can enable the hardened runtime by clicking on the “+Capability” button above the “Signing” section and selecting “Hardened Runtime”.

Archive and export the binary

Choose “Archive” from the “Product” menu to build and create an archive. It will appear in the “Organizer” window. When that window does not open automatically, you can access it from the “Window” menu.

To export the binary product, select the latest archive and click on the “Distribute Content” button on the right. Choose “Built Products” as the method of distribution. Click “Next.” Choose a location to save the build products to.

This will create a directory with the project name and a timestamp in the chosen location. When you look inside this directory, you will see a “Products” directory and within it the binary in a /usr/local/bin/ directory hierarchy.

/usr/local/bin is the default location for command line tools in the Command Line Tool project template. It suits me fine most of the time, but you can change it by modifying the ‘Installation Directory’ build setting in Xcode and re-building the archive.

Build the installer package

Command Line Tools can be signed, but not directly notarized. You can however notarize a pkg file containing the Command Line Tool. Also, it is much easier for users and administrators to install your tool when it comes in a proper installation package.

We can use the Products directory as our payload to build the installer package:

% pkgbuild --root "hello 2021-mm-dd hh-mm-ss/Products" \
           --identifier "com.example.hello" \
           --version "1.0" \
           --install-location "/" \
           --sign "Developer ID Installer: Name (ABCD123456)" \
           hello-1.0.pkg

I have broken the command into multiple lines for clarity, you can enter the command in one line without the end-of-line backslashes \. You want to replace the values for the identifier, version and signing certificate with your data.

This will build an installer package which would install your binary on the target system. You should inspect the pkg file with Pacifist or Suspicious Package and do a test install on a test system to verify everything works.

If you want to learn more about installer packages and pkgbuild read my book “Packaging for Apple Administrators.”

Notarizing the Installer Package

Now we get to the new, most interesting part. We will notarize the newly-created installer package with notarytool:

% xcrun notarytool submit hello-1.0.pkg \
                   --keychain-profile "notary-scriptingosx" \
                   --wait

This is amazingly less effort than what we needed to do previously with the altool command. We give the filename of the archive we want to submit, the keychain profile with our credentials, and the --wait option.

notarytool will upload the file, give us a submission id, and then wait for the returned status from the Notary service. You can follow the output for the details.

You will also notice that notarytool uploads the pkg much faster than the previous altool workflow.

You can also drop the --wait option. Then the tool will submit the file and exit without waiting for a response. You can then use the info or log verbs with the submission id to get the status later. The Notary service does not seem to send emails anymore when the notarization check is complete.

There is also a --webhook option mentioned in the WWDC session which will make the Notary service call back to a webhook when the notarization is done. I have not seen any documentation on the details of this, though.

Finishing touch: stapler

Before you distribute the pkg, you can and should ‘staple’ the notarization before distributing it. This extra step will download the notarization information from Apple’s servers and attach it to the pkg. This is not mandatory, but will save the Gatekeeper service on the client an extra step when it verifies the pkg.

To do this, use the eponymous stapler tool:

% xcrun stapler staple hello-1.0.pkg

You can then verify that everything works with spctl:

% spctl --assess -vv --type install hello-1.0.pkg

Automation with Xcode

These steps are much simplified compared to the previous workflow. If you only build for distribution occasionally it would not be a big burden to do these steps manually.

Nevertheless, automating these steps saves effort and removes much pontential for errors.

When I wrote the previous post, I had not been able to figure out how all the pieces could work together to automate with a Xcode ‘Run Script’ as part of the normal “Archive” process. With the new tool and some inspiration from this developer article I have gotten this to work now.

In the project’s build settings, search for “Marketing Version” and set it to the version you want to use. Remember to update this entry for future updates as well. (You can use agvtool for this, but that is a topic for a different post.)

In Xcode, choose “Edit Scheme…” from the “Scheme” submenu in the “Project” menu. In the pane that opens, make sure the commnad line tool binary is selected at the top. Then expand the “Archive” section in the list on the left and select “Post-actions” in the expanded area. Use the “+” button at the bottom of the area to add a “New Run Script Action.”

Select the binary (again) in the popup next to “Provide build settings from”. Then paste the following in the code field:

With this post-action script in place, every “Archive” action will then also create a pkg in the project folder, submit it for notarization and staple the pkg. Since Xcode doesn’t show the output of post-action scripts, the script logs its output to a notary.log file, also in the project folder. Check that for success or failures. The notarization step takes a while after the “Archive” is complete, so you may have to wait a bit.

If you don’t want to run this workflow on every Archive, you can create a new scheme with this post-action script, then you can choose the scheme, before you do the “Archive” action.

Conclusion

The new notarytool included with Xcode 13 (beta) is a huge step up from the previous altool based workflows. It is much simpler and faster. You should start testing the tool now and move your workflows when possible.

Weekly News Summary for Admins — 2021–07–02

We made it halfway through 2021 already… How did that happen?

If you were hoping things would calm down for the summer, we got new macOS 11.5/iOS 14.7 betas and macOS 12 Monterey beta2! Apple has also released iOS 15 and macOS 12 beta2 as in the public beta program. You are testing and providing feedback to Apple and the other software vendors, right?


(Sponsor: Mosyle)

Mosyle Fuse logo

The Fusion of Apple MDM, Identity, Patching & Security.

Mosyle Fuse is the first and only product to bring a perfect blend of an Enterprise-grade MDM, an innovative solution for macOS Identity Management, automated application installation and patching, and purpose-built multi-layer endpoint security, all specially designed for Apple devices used at work at a price point that’s almost unexplainable.

Click here to learn more!


If you would rather get the weekly newsletter by email, you can subscribe to the Scripting OS X Weekly Newsletter here!! (Same content, delivered to your Inbox once a week.)

📰News and Opinion

🐟macOS 12 Monterey and iOS 15

🐦MacAdmins on Twitter

  • Anthony Reimer: “At the suggestion of @scriptingosx, I’ve added T2 Chip information to my Mac Obsolescence Chart for MacAdmins (useful for a number of reasons, including which macOS 12 Macs can be assigned to ABM/ASM/DEP using Apple Configurator for iPhone in iOS 15).”

🔐Security and Privacy

🔨Support and HowTos

🤖Scripting and Automation

🍏Apple Support

♻️Updates and Releases

🎧To Listen

🎈Just for Fun

📚Support

If you are enjoying what you are reading here, please spread the word and recommend it to another Mac Admin!

If you want to support me and this website even further, then consider buying one (or all) of my books. It’s like a subscription fee, but you also get a useful book or two extra!