Scripting OS X

Anthony Reimer and I had a lot of fun sending an AirTag across the Atlantic. Now we get to the experience of being on the receiving side.

“AirTag Found Moving With You”

The “Find My” network warns you when an unknown AirTag is moving with you. This is to prevent tracking people without their approval. Since Anthony had registered the AirTag on his account to track its travel, this was a perfect opportunity to test this situation.

I just dropped the AirTag in the front pocket of my backpack and went about my business. This is the backpack I use to transport odds and ends, especially groceries, so it nearly always goes where I go. At first the backpack remained immobile in my house for the afternoon and then overnight. The next morning, we took a trip to the Leiden Farmers’ Market when I got the warning that an AirTag was moving with me.

It is interesting (but makes sense) that the warning didn’t come until I actually started moving with the AirTag. The tag is not really tracking you when it is just sitting around. But once I was on the go (together with the “strange” AirTag) I was warned fairly quickly: after about 2.5 hours. My wife also got the same warning on her iPhone, which should not be surprising, since we were walking together.

The AirTag is supposed to eventually make a sound when it is separated from its owner, but it never got to that phase in our testing, or I did not hear it.

The iPhone showed me a map where my iPhone had detected the “strange” AirTag and offered the option to play a sound on the AirTag to help locate it. Presumably, when someone tracks you without your knowledge, the AirTag would be hidden somewhere nearby and the sound will help you find it.

You can tell your phone to ignore this particular AirTag, presumably after you have checked with your companions who are travelling with you or because you are carrying a borrowed, tagged item.

The app also shows the serial number of the AirTag and the last four digits of the phone number it is registered to. These digits should allow you to identify the owner, when you know them, but maintain their anonymity when you don’t.

You can also get instructions to disable the AirTag. This will show instructions to open it and remove the battery with a simple but effective animation. (AirTags open real easy, given how small they are and how solidly sealed they seem when closed. This is really impressive engineering.)

I can see a possible downside here. The disabling process requires you to actually find the AirTag. If someone manages to hide the AirTag in way that you cannot find or access it physically, you cannot disable it. This might be harder than I imagine, because shielding the AirTag in a way that muffles the sound sufficiently might also shield the Bluetooth transmissions, which prevent the tracking in the first place. More experimentation will be needed here.

Lost Mode

Now that the covert tracking had been tested, Anthony set the AirTag to lost mode on his account. It took a few minutes for that change to propagate through the network. With lost mode enabled, I could call up his contact information (the owner can choose whether to show an email or the phone number) from the AirTag on my phone, just by tapping my phone to the AirTag.

Anthony also tried to play a sound on the AirTag, which was more than 7000km away from him. This did not work. It seems that playing the sound requires a local bluetooth connection to the AirTag. Since you would likely not be able to hear the sound when you are out of bluetooth range, and could use this to ‘terrorize’ someone (intentionally or not) in the middle of the night, I think this a reasonable limitation.

Transferring the AirTag

With all our testing done it was time for Anthony to remove the AirTag from his account, so that I could add it to my account. The interface for that in the Find My application is very straightforward.

He did, however, get an error that the iPhone could not “find” the AirTag. We presume his iPhone tried to connect to the AirTag over local bluetooth to let it “know” it was removed.

After Anthony had removed the device from his account, I tried to set it up on mine. This did not immediately work. Even after waiting for a few hours, my phone would not recognize the AirTag as new.

I then followed the instructions in this support article to reset the AirTag. It’s a bit tedious as you have to remove and replace the battery five times in a row. I figured out you don’t have to actually close the lid five times, just taking out the battery and putting it back in its place is sufficient. (there are magnets in the AirTag that seem to hold the batttery in place) After the reset process, the AirTag appeared immediately for setup on my phone and I could add it to my iCloud account.

Conclusion

Overall, the user experience for both the “Moving with you” and “Lost Mode” workflows are well thought through and kept clear and simple. Apple has good support articles for reference.

Many thanks to the comittee of MacDeployment and their sponsors that provided AirTags to all the speakers. And thanks to Anthony, who was game when I suggested that sending and tracking an AirTag across the Atlantic would be the “most fun” way to get them to me. Hope you found our experiments interesting, as well!

Right now, the AirTag has returned to my backpack. This seems reasonable since it stores my wallet and keys when I leave the house. I also want to test attaching an AirTag to my bike. I believe that bike thieves will quickly catch on to AirTags, so I don’t have high hopes for it to be useful as theft prevention. But an AirTag on the bike should be very useful to find my bike again in one of the typical Dutch bike parking areas among thousands of other bikes.

When Apple introduced notarization with Catalina, I published a post describing how to notarize a command line tool. At WWDC this year, Apple introduced updates to this process with Xcode 13 (currently in beta). Most importantly, there is a new command line tool called notarytool.

While the previous, altool-based, workflow still works in Xcode 13, there are many advantages to the new notarytool which makes its use much simpler.

Apple has documented this tool in a WWDC21 session and some developer articles, in addition we got some great information through the twitter account of one of the engineers, and Howard Oakley has already written a post as well:

• WWDC 21: “Faster and Simpler notarization for Mac apps“
• Apple Developer : Customizing the Notarization Workflow
• Rosyna Keller: “Let’s briefly talk about the notarytool, a new, dedicated command line tool for notarization that replaces altool (for notarization). “ (Thread)
• Howard Oakley: Will changes to notarization make any difference?

Update 2023-08-28: If you prefer to use Swift Package Manager to build command line tools, you can find instructions to package and notarize an SPM executable in this post.

What you need

• Apple Developer Account (Personal or Enterprise, the free account does not provide the right certificates, nor access to the Xcode beta)
• Xcode 13 (currently available as beta from the Apple Developer portal)
• Developer ID Certificates
• Application Specific Password for your Developer account
• A command line tool project in Xcode

When you are building tools for macOS, you should have most of these already. We already covered these in the previous post, but to keep things in one place, I will cover them again, here.

Apple Developer Account

You need either the paid membership in the Apple Developer Program or be invited to an Apple Developer Enterprise Program team with access to the proper certificates.

You cannot get the required certificates with a free Apple Developer account, unless you are member of a team that provides access.

Xcode 13 (beta)

Until the full version of Xcode 13 is released, you can get Xcode 13 beta from the beta downloads page on the Apple Developer Portal.

Once it is released (usually when iOS is released) you will be able to download it from the Mac App Store, as well.

Xcode 13 requires macOS Big Sur 11.3 or higher. According to this tweet from Rosyna Keller, notarytool can be extracted and run on macOS Catalina 10.15.7 and higher.

You can run the notarytool binary through xcrun:

% xcrun notarytool --help

If you need to extract the binary you can find where is stored on disk with:

% xcrun --find notarytool
/Applications/Xcode-beta.app/Contents/Developer/usr/bin/notarytool

Developer ID Certificates

There are multiple certificates you can get from the Developer Program. By default you get a ‘Mac Developer’ certificate, which you can use for building and testing your own app locally.

To distribute binaries (apps and command line tools) outside of the App Store, you need a ‘Developer ID Application’ certificate. To sign installer packages for distribution outside of the Mac App Store, you need a ‘Developer ID Installer’ certificate.

We will need both types of Developer ID certificates, the first to sign the command line tool and the second to sign and notarize the installer package.

If you have not created these yet, you can do so in Xcode or in the Developer Portal. If you already have the certificates but on a different Mac, you need to export them and re-import them on the new Mac. Creating new certificates might invalidate the existing certificates! So beware.

Once you have created or imported the certificates on your work machine, you can verify their presence in the Terminal with:

% security find-identity -p basic -v

This command will list all available certificates on this Mac. Check that you can see the ‘Developer ID Application’ and ‘Developer ID Installer’ certificates. If you are a member of multiple teams, you may see multiple certificates for each team.

You can later identify the certificates (or ‘identities’) by the long hex number or by the descriptive name, e.g. "Developer ID Installer: Armin Briegel (ABCD123456)"

The ten character code at the end of the name is your Developer Team ID. Make a note of it, we will need it later. If you are a member of multiple developer teams, you can have multiple Developer ID certificates and the team ID will help you distinguish them.

Application Specific Password for your Developer Account

Apple requires Developer Accounts to be protected with two-factor authentication. To allow automated workflows which require authentication, you can create application specific passwords.

Note: If you followed the previous post’s instructions to store an application specific password for altool in the Keychain, you can extract that and re-use it for notarytool or create a new app-specific password.

Create a new application specific password in Apple ID portal for your developer account. Give it a name including notarytool so you know what you are using this for.

• Apple Support: Using app-specific passwords

You will only be shown the password when you create it.

You can use notarytool to store the credentials in a keychain item, in a format that notarytool can read later.

% xcrun notarytool store-credentials --apple-id "name@example.com" --team-id "ABCD123456"

This process stores your credentials securely in the Keychain. You reference these credentials later using a profile name.

Profile name:
notary-example.com
Password for name@example.com: 
Validating your credentials...
Success. Credentials validated.
Credentials saved to Keychain.
To use them, specify `--keychain-profile "notary-example.com"`

The --store-credentials option will prompt for a profile name. You will need this name to retrieve the information later. Then it interactively prompts for the password associated with the given Apple Developer ID. Enter the application specific password here.

The credentials will be stored in the Keychain in an item named com.apple.gke.notary.tool. But you don’t really have to worry about that since notarytool will retrieve the credentials when you add the --keychain-profile "notary-example.com" option. (You can abbreviate the --keychain-profile with -p.)

If you are using iCloud Keychain, the credentials will be stored there, so they will be available to all other Macs you are using iCloud Keychain with. If you prefer, you can store the credentials in a specific (non-iCloud) keychain file with the --keychain option.

The Team ID is usually the 10-digit code which is also the certificates. However, in some cases the Team ID is different. You can can look-up Team IDs in the “Membership” area of the developer portal or with this altool command:

% xcrun altool --list-providers -u "name@example.com" -p "@keychain:<ITEM_NAME>"

(Thanks to ‘mhp’ for sharing this.)

You can also use an App Store Connect API key as an authentication option with notarytool. You can read notarytool‘s man page for details.

A Command Line Tool Project

You may already have a project to create a command line in Xcode. If you don’t have one, or just want a new one to experiment, you can just create a new project in Xcode and choose the ‘Command Line Tool’ template from ‘macOS’ section in the picker. The template creates a simple “Hello, world” tool, which you can use to test the notarization process.

My sample project for this article will be named “hello.”

Preparing the Xcode Project

The default settings in the ‘Command Line Tool’ project are suitable for building and testing the tool on your Mac, but need some changes to create a distributable tool.

The preparation in Xcode 13 diverges significantly from the steps required in the previous post. If you have created the project in earlier versions of Xcode, more configuration may be necessary.

Choosing the proper signing certificates

Before you can notarize the command line tool, it needs to be signed with the correct certificates.

1. in Xcode, select the blue project icon in the left sidebar
2. select the black “terminal” icon with your project’s name under the “Targets” list entry
3. make sure the ‘Signing & Certificates’ tab is selected
4. under ‘Signing’ disable ‘Automatically manage signing’
5. choose your Team
6. enter a bundle identifier for the binary
7. choose ‘Developer ID Application‘ as the Signing Certificate

Hardened Runtime

Having the “Hardened Runtime” enabled is a requirement for notarization. When you create a new project in Xcode 13, the hardened runtime will be enabled by default. When you see the “Hardened Runtime” section under the “Signing” section, it is enabled.

When you are working with a older project, and do not see the “Hardened Runtime” section, you can enable the hardened runtime by clicking on the “+Capability” button above the “Signing” section and selecting “Hardened Runtime”.

Archive and export the binary

Choose “Archive” from the “Product” menu to build and create an archive. It will appear in the “Organizer” window. When that window does not open automatically, you can access it from the “Window” menu.

To export the binary product, select the latest archive and click on the “Distribute Content” button on the right. Choose “Built Products” as the method of distribution. Click “Next.” Choose a location to save the build products to.

This will create a directory with the project name and a timestamp in the chosen location. When you look inside this directory, you will see a “Products” directory and within it the binary in a /usr/local/bin/ directory hierarchy.

/usr/local/bin is the default location for command line tools in the Command Line Tool project template. It suits me fine most of the time, but you can change it by modifying the ‘Installation Directory’ build setting in Xcode and re-building the archive.

Build the installer package

Command Line Tools can be signed, but not directly notarized. You can however notarize a pkg file containing the Command Line Tool. Also, it is much easier for users and administrators to install your tool when it comes in a proper installation package.

We can use the Products directory as our payload to build the installer package:

% pkgbuild --root "hello 2021-mm-dd hh-mm-ss/Products" \
           --identifier "com.example.hello" \
           --version "1.0" \
           --install-location "/" \
           --sign "Developer ID Installer: Name (ABCD123456)" \
           hello-1.0.pkg

I have broken the command into multiple lines for clarity, you can enter the command in one line without the end-of-line backslashes \. You want to replace the values for the identifier, version and signing certificate with your data.

This will build an installer package which would install your binary on the target system. You should inspect the pkg file with Pacifist or Suspicious Package and do a test install on a test system to verify everything works.

If you want to learn more about installer packages and pkgbuild read my book “Packaging for Apple Administrators.”

Notarizing the Installer Package

Now we get to the new, most interesting part. We will notarize the newly-created installer package with notarytool:

% xcrun notarytool submit hello-1.0.pkg \
                   --keychain-profile "notary-scriptingosx" \
                   --wait

This is amazingly less effort than what we needed to do previously with the altool command. We give the filename of the archive we want to submit, the keychain profile with our credentials, and the --wait option.

notarytool will upload the file, give us a submission id, and then wait for the returned status from the Notary service. You can follow the output for the details.

You will also notice that notarytool uploads the pkg much faster than the previous altool workflow.

You can also drop the --wait option. Then the tool will submit the file and exit without waiting for a response. You can then use the info or log verbs with the submission id to get the status later. The Notary service does not seem to send emails anymore when the notarization check is complete.

There is also a --webhook option mentioned in the WWDC session which will make the Notary service call back to a webhook when the notarization is done. I have not seen any documentation on the details of this, though.

Finishing touch: stapler

Before you distribute the pkg, you can and should ‘staple’ the notarization before distributing it. This extra step will download the notarization information from Apple’s servers and attach it to the pkg. This is not mandatory, but will save the Gatekeeper service on the client an extra step when it verifies the pkg.

To do this, use the eponymous stapler tool:

% xcrun stapler staple hello-1.0.pkg

You can then verify that everything works with spctl:

% spctl --assess -vv --type install hello-1.0.pkg

Automation with Xcode

These steps are much simplified compared to the previous workflow. If you only build for distribution occasionally it would not be a big burden to do these steps manually.

Nevertheless, automating these steps saves effort and removes much pontential for errors.

When I wrote the previous post, I had not been able to figure out how all the pieces could work together to automate with a Xcode ‘Run Script’ as part of the normal “Archive” process. With the new tool and some inspiration from this developer article I have gotten this to work now.

In the project’s build settings, search for “Marketing Version” and set it to the version you want to use. Remember to update this entry for future updates as well. (You can use agvtool for this, but that is a topic for a different post.)

In Xcode, choose “Edit Scheme…” from the “Scheme” submenu in the “Project” menu. In the pane that opens, make sure the commnad line tool binary is selected at the top. Then expand the “Archive” section in the list on the left and select “Post-actions” in the expanded area. Use the “+” button at the bottom of the area to add a “New Run Script Action.”

Select the binary (again) in the popup next to “Provide build settings from”. Then paste the following in the code field:

	# Developer ID Installer cert name
	sign_cert="Developer ID Installer: Name (ABCD123456)"
	# profile name used with `notarytool --store-credentials`
	credential_profile="notary-example.com"
	# data from build settings
	pkg_name="$PRODUCT_NAME"
	identifier="$PRODUCT_BUNDLE_IDENTIFIER"
	version="$MARKETING_VERSION"
	PKG_PATH="$SRCROOT/$pkg_name-$version.pkg"
	NOTARY_LOG="$SRCROOT/notary.log"
	echo "Packaging and Notarizing '$pkg_name', version: $version" > "$NOTARY_LOG"
	date +"%F %T" >> "$NOTARY_LOG"
	echo >> "$NOTARY_LOG"
	# usually use `xcodebuild -exportArchive` to get
	# the product out of the archive. However, this does not work
	# with a command line tool, so we are going direct
	PKG_ROOT="$ARCHIVE_PATH/Products/"
	# create the pkg
	pkgbuild --root "$PKG_ROOT" \
	--identifier "$identifier" \
	--version "$version" \
	--install-location "/" \
	--sign "$sign_cert" \
	"$PKG_PATH" >> "$NOTARY_LOG" 2>&1
	echo >> "$NOTARY_LOG"
	# notarize
	xcrun notarytool submit "$PKG_PATH" \
	--keychain-profile "$credential_profile" \
	--wait >> "$NOTARY_LOG" 2>&1
	echo >> "$NOTARY_LOG"
	# staple
	xcrun stapler staple "$PKG_PATH" >> "$NOTARY_LOG" 2>&1
	# reveal in Finder
	open -R "$PKG_PATH"

view raw Build Package and Notarize hosted with by GitHub

With this post-action script in place, every “Archive” action will then also create a pkg in the project folder, submit it for notarization and staple the pkg. Since Xcode doesn’t show the output of post-action scripts, the script logs its output to a notary.log file, also in the project folder. Check that for success or failures. The notarization step takes a while after the “Archive” is complete, so you may have to wait a bit.

If you don’t want to run this workflow on every Archive, you can create a new scheme with this post-action script, then you can choose the scheme, before you do the “Archive” action.

Conclusion

The new notarytool included with Xcode 13 (beta) is a huge step up from the previous altool based workflows. It is much simpler and faster. You should start testing the tool now and move your workflows when possible.