On a personal note: today is my first day working at Jamf as a Consulting Engineer.
I have been working with Jamf for more than a decade. Back then it was called Casper. Over the years, I have experienced the product and the company as a partner, as a customer and admin, and as a consultant. Working from the inside seemed like a logical next step. I will be joining a great team where I already know many people and I am looking forward to getting to know everyone else!
But don’t be afraid, this weblog and the weekly newsletter will continue as usual!
The “Find My” network warns you when an unknown AirTag is moving with you. This is to prevent tracking people without their approval. Since Anthony had registered the AirTag on his account to track its travel, this was a perfect opportunity to test this situation.
I just dropped the AirTag in the front pocket of my backpack and went about my business. This is the backpack I use to transport odds and ends, especially groceries, so it nearly always goes where I go. At first the backpack remained immobile in my house for the afternoon and then overnight. The next morning, we took a trip to the Leiden Farmers’ Market when I got the warning that an AirTag was moving with me.
It is interesting (but makes sense) that the warning didn’t come until I actually started moving with the AirTag. The tag is not really tracking you when it is just sitting around. But once I was on the go (together with the “strange” AirTag) I was warned fairly quickly: after about 2.5 hours. My wife also got the same warning on her iPhone, which should not be surprising, since we were walking together.
The AirTag is supposed to eventually make a sound when it is separated from its owner, but it never got to that phase in our testing, or I did not hear it.
The iPhone showed me a map where my iPhone had detected the “strange” AirTag and offered the option to play a sound on the AirTag to help locate it. Presumably, when someone tracks you without your knowledge, the AirTag would be hidden somewhere nearby and the sound will help you find it.
You can tell your phone to ignore this particular AirTag, presumably after you have checked with your companions who are travelling with you or because you are carrying a borrowed, tagged item.
The app also shows the serial number of the AirTag and the last four digits of the phone number it is registered to. These digits should allow you to identify the owner, when you know them, but maintain their anonymity when you don’t.
You can also get instructions to disable the AirTag. This will show instructions to open it and remove the battery with a simple but effective animation. (AirTags open real easy, given how small they are and how solidly sealed they seem when closed. This is really impressive engineering.)
I can see a possible downside here. The disabling process requires you to actually find the AirTag. If someone manages to hide the AirTag in way that you cannot find or access it physically, you cannot disable it. This might be harder than I imagine, because shielding the AirTag in a way that muffles the sound sufficiently might also shield the Bluetooth transmissions, which prevent the tracking in the first place. More experimentation will be needed here.
Lost Mode
Now that the covert tracking had been tested, Anthony set the AirTag to lost mode on his account. It took a few minutes for that change to propagate through the network. With lost mode enabled, I could call up his contact information (the owner can choose whether to show an email or the phone number) from the AirTag on my phone, just by tapping my phone to the AirTag.
Anthony also tried to play a sound on the AirTag, which was more than 7000km away from him. This did not work. It seems that playing the sound requires a local bluetooth connection to the AirTag. Since you would likely not be able to hear the sound when you are out of bluetooth range, and could use this to ‘terrorize’ someone (intentionally or not) in the middle of the night, I think this a reasonable limitation.
Transferring the AirTag
With all our testing done it was time for Anthony to remove the AirTag from his account, so that I could add it to my account. The interface for that in the Find My application is very straightforward.
He did, however, get an error that the iPhone could not “find” the AirTag. We presume his iPhone tried to connect to the AirTag over local bluetooth to let it “know” it was removed.
After Anthony had removed the device from his account, I tried to set it up on mine. This did not immediately work. Even after waiting for a few hours, my phone would not recognize the AirTag as new.
I then followed the instructions in this support article to reset the AirTag. It’s a bit tedious as you have to remove and replace the battery five times in a row. I figured out you don’t have to actually close the lid five times, just taking out the battery and putting it back in its place is sufficient. (there are magnets in the AirTag that seem to hold the batttery in place) After the reset process, the AirTag appeared immediately for setup on my phone and I could add it to my iCloud account.
Conclusion
Overall, the user experience for both the “Moving with you” and “Lost Mode” workflows are well thought through and kept clear and simple. Apple has good support articles for reference.
Many thanks to the comittee of MacDeployment and their sponsors that provided AirTags to all the speakers. And thanks to Anthony, who was game when I suggested that sending and tracking an AirTag across the Atlantic would be the “most fun” way to get them to me. Hope you found our experiments interesting, as well!
Right now, the AirTag has returned to my backpack. This seems reasonable since it stores my wallet and keys when I leave the house. I also want to test attaching an AirTag to my bike. I believe that bike thieves will quickly catch on to AirTags, so I don’t have high hopes for it to be useful as theft prevention. But an AirTag on the bike should be very useful to find my bike again in one of the typical Dutch bike parking areas among thousands of other bikes.
When Apple introduced notarization with Catalina, I published a post describing how to notarize a command line tool. At WWDC this year, Apple introduced updates to this process with Xcode 13 (currently in beta). Most importantly, there is a new command line tool called notarytool.
While the previous, altool-based, workflow still works in Xcode 13, there are many advantages to the new notarytool which makes its use much simpler.
Apple has documented this tool in a WWDC21 session and some developer articles, in addition we got some great information through the twitter account of one of the engineers, and Howard Oakley has already written a post as well:
Update 2023-08-28: If you prefer to use Swift Package Manager to build command line tools, you can find instructions to package and notarize an SPM executable in this post.
What you need
Apple Developer Account (Personal or Enterprise, the free account does not provide the right certificates, nor access to the Xcode beta)
Xcode 13 (currently available as beta from the Apple Developer portal)
Developer ID Certificates
Application Specific Password for your Developer account
A command line tool project in Xcode
When you are building tools for macOS, you should have most of these already. We already covered these in the previous post, but to keep things in one place, I will cover them again, here.
There are multiple certificates you can get from the Developer Program. By default you get a ‘Mac Developer’ certificate, which you can use for building and testing your own app locally.
To distribute binaries (apps and command line tools) outside of the App Store, you need a ‘Developer ID Application’ certificate. To sign installer packages for distribution outside of the Mac App Store, you need a ‘Developer ID Installer’ certificate.
We will need both types of Developer ID certificates, the first to sign the command line tool and the second to sign and notarize the installer package.
Once you have created or imported the certificates on your work machine, you can verify their presence in the Terminal with:
% security find-identity -p basic -v
This command will list all available certificates on this Mac. Check that you can see the ‘Developer ID Application’ and ‘Developer ID Installer’ certificates. If you are a member of multiple teams, you may see multiple certificates for each team.
You can later identify the certificates (or ‘identities’) by the long hex number or by the descriptive name, e.g. "Developer ID Installer: Armin Briegel (ABCD123456)"
The ten character code at the end of the name is your Developer Team ID. Make a note of it, we will need it later. If you are a member of multiple developer teams, you can have multiple Developer ID certificates and the team ID will help you distinguish them.
Application Specific Password for your Developer Account
Apple requires Developer Accounts to be protected with two-factor authentication. To allow automated workflows which require authentication, you can create application specific passwords.
Note: If you followed the previous post’s instructions to store an application specific password for altool in the Keychain, you can extract that and re-use it for notarytool or create a new app-specific password.
Create a new application specific password in Apple ID portal for your developer account. Give it a name including notarytool so you know what you are using this for.
You will only be shown the password when you create it.
You can use notarytool to store the credentials in a keychain item, in a format that notarytool can read later.
% xcrun notarytool store-credentials --apple-id "name@example.com" --team-id "ABCD123456"
This process stores your credentials securely in the Keychain. You reference these credentials later using a profile name.
Profile name:
notary-example.com
Password for name@example.com:
Validating your credentials...
Success. Credentials validated.
Credentials saved to Keychain.
To use them, specify `--keychain-profile "notary-example.com"`
The --store-credentials option will prompt for a profile name. You will need this name to retrieve the information later. Then it interactively prompts for the password associated with the given Apple Developer ID. Enter the application specific password here.
The credentials will be stored in the Keychain in an item named com.apple.gke.notary.tool. But you don’t really have to worry about that since notarytool will retrieve the credentials when you add the --keychain-profile "notary-example.com" option. (You can abbreviate the --keychain-profile with -p.)
If you are using iCloud Keychain, the credentials will be stored there, so they will be available to all other Macs you are using iCloud Keychain with. If you prefer, you can store the credentials in a specific (non-iCloud) keychain file with the --keychain option.
The Team ID is usually the 10-digit code which is also the certificates. However, in some cases the Team ID is different. You can can look-up Team IDs in the “Membership” area of the developer portal or with this altool command:
You can also use an App Store Connect API key as an authentication option with notarytool. You can read notarytool‘s man page for details.
A Command Line Tool Project
You may already have a project to create a command line in Xcode. If you don’t have one, or just want a new one to experiment, you can just create a new project in Xcode and choose the ‘Command Line Tool’ template from ‘macOS’ section in the picker. The template creates a simple “Hello, world” tool, which you can use to test the notarization process.
My sample project for this article will be named “hello.”
Preparing the Xcode Project
The default settings in the ‘Command Line Tool’ project are suitable for building and testing the tool on your Mac, but need some changes to create a distributable tool.
The preparation in Xcode 13 diverges significantly from the steps required in the previous post. If you have created the project in earlier versions of Xcode, more configuration may be necessary.
Choosing the proper signing certificates
Before you can notarize the command line tool, it needs to be signed with the correct certificates.
in Xcode, select the blue project icon in the left sidebar
select the black “terminal” icon with your project’s name under the “Targets” list entry
make sure the ‘Signing & Certificates’ tab is selected
under ‘Signing’ disable ‘Automatically manage signing’
choose your Team
enter a bundle identifier for the binary
choose ‘Developer ID Application‘ as the Signing Certificate
Hardened Runtime
Having the “Hardened Runtime” enabled is a requirement for notarization. When you create a new project in Xcode 13, the hardened runtime will be enabled by default. When you see the “Hardened Runtime” section under the “Signing” section, it is enabled.
When you are working with a older project, and do not see the “Hardened Runtime” section, you can enable the hardened runtime by clicking on the “+Capability” button above the “Signing” section and selecting “Hardened Runtime”.
Archive and export the binary
Choose “Archive” from the “Product” menu to build and create an archive. It will appear in the “Organizer” window. When that window does not open automatically, you can access it from the “Window” menu.
To export the binary product, select the latest archive and click on the “Distribute Content” button on the right. Choose “Built Products” as the method of distribution. Click “Next.” Choose a location to save the build products to.
This will create a directory with the project name and a timestamp in the chosen location. When you look inside this directory, you will see a “Products” directory and within it the binary in a /usr/local/bin/ directory hierarchy.
/usr/local/bin is the default location for command line tools in the Command Line Tool project template. It suits me fine most of the time, but you can change it by modifying the ‘Installation Directory’ build setting in Xcode and re-building the archive.
Build the installer package
Command Line Tools can be signed, but not directly notarized. You can however notarize a pkg file containing the Command Line Tool. Also, it is much easier for users and administrators to install your tool when it comes in a proper installation package.
We can use the Products directory as our payload to build the installer package:
I have broken the command into multiple lines for clarity, you can enter the command in one line without the end-of-line backslashes \. You want to replace the values for the identifier, version and signing certificate with your data.
This will build an installer package which would install your binary on the target system. You should inspect the pkg file with Pacifist or Suspicious Package and do a test install on a test system to verify everything works.
This is amazingly less effort than what we needed to do previously with the altool command. We give the filename of the archive we want to submit, the keychain profile with our credentials, and the --wait option.
notarytool will upload the file, give us a submission id, and then wait for the returned status from the Notary service. You can follow the output for the details.
You will also notice that notarytool uploads the pkg much faster than the previous altool workflow.
You can also drop the --wait option. Then the tool will submit the file and exit without waiting for a response. You can then use the info or log verbs with the submission id to get the status later. The Notary service does not seem to send emails anymore when the notarization check is complete.
There is also a --webhook option mentioned in the WWDC session which will make the Notary service call back to a webhook when the notarization is done. I have not seen any documentation on the details of this, though.
Finishing touch: stapler
Before you distribute the pkg, you can and should ‘staple’ the notarization before distributing it. This extra step will download the notarization information from Apple’s servers and attach it to the pkg. This is not mandatory, but will save the Gatekeeper service on the client an extra step when it verifies the pkg.
To do this, use the eponymous stapler tool:
% xcrun stapler staple hello-1.0.pkg
You can then verify that everything works with spctl:
% spctl --assess -vv --type install hello-1.0.pkg
Automation with Xcode
These steps are much simplified compared to the previous workflow. If you only build for distribution occasionally it would not be a big burden to do these steps manually.
Nevertheless, automating these steps saves effort and removes much pontential for errors.
When I wrote the previous post, I had not been able to figure out how all the pieces could work together to automate with a Xcode ‘Run Script’ as part of the normal “Archive” process. With the new tool and some inspiration from this developer article I have gotten this to work now.
In the project’s build settings, search for “Marketing Version” and set it to the version you want to use. Remember to update this entry for future updates as well. (You can use agvtool for this, but that is a topic for a different post.)
In Xcode, choose “Edit Scheme…” from the “Scheme” submenu in the “Project” menu. In the pane that opens, make sure the commnad line tool binary is selected at the top. Then expand the “Archive” section in the list on the left and select “Post-actions” in the expanded area. Use the “+” button at the bottom of the area to add a “New Run Script Action.”
Select the binary (again) in the popup next to “Provide build settings from”. Then paste the following in the code field:
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
With this post-action script in place, every “Archive” action will then also create a pkg in the project folder, submit it for notarization and staple the pkg. Since Xcode doesn’t show the output of post-action scripts, the script logs its output to a notary.log file, also in the project folder. Check that for success or failures. The notarization step takes a while after the “Archive” is complete, so you may have to wait a bit.
If you don’t want to run this workflow on every Archive, you can create a new scheme with this post-action script, then you can choose the scheme, before you do the “Archive” action.
Conclusion
The new notarytool included with Xcode 13 (beta) is a huge step up from the previous altool based workflows. It is much simpler and faster. You should start testing the tool now and move your workflows when possible.
While attending WWDC and MacDevOps YVR last week – or at least attempting to – I realized that you can get real jet lag from virtual conferences.
Many people are catching up to the news from WWDC last week with many posts reacting to and/or summarizing the news. We also a patch for iOS 12 and and new macOS 11.5/iOS 14.7 betas.
(Sponsor: Mosyle)
Free Remote Scripting with Mosyle Business FREE
From running scripts remotely to full Mobile Device Management (MDM) for macOS, iOS and tvOS, Mosyle Business FREE provides Apple enterprise customers with the complete MDM feature set of Mosyle Business PREMIUM for up to 30 devices at no charge.
I’d like to thank Mosyle for being the new sponsor for this news summary! I have been watching what they have been doing in the macOS MDM space and believe they are a great fit as a sponsor.
Morten Just: “Go To Folder – ⌘⇧G in Finder got its first update in ~15 years with Monterey. Spotlight-style UI, and you can search for any folder https://t.co/jfGe1Z1RAw” (video)
Rich Trouton: “The more I look at macOS Monterey, the more that I’m convinced it is kindred to Mac OS X Snow Leopard – an OS focused on fixing existing issues and beneficial improvements to previously introduced features. This is a Good Thing and I look forward to the fall release.”
Anthony Reimer: “Reminded of the model-specific features of macOS Monterey by @howardnoakley and @TidBITS, I’ve updated my Mac Hardware/Software Obsolescence Chart (yes, for the 3rd time in a week) with info on support for Universal Control and AirPlay for Mac.”
mikeymikey: “Almost every scripted language in macOS provides a warning if you execute it interactively. Tcl, ruby, perl, you name it.”
If you want to support me and this website even further, then consider buying one (or all) of my books. It’s like a subscription fee, but you also get a useful book or two extra!
So, WWDC started yesterday and Apple will release more than 200 sessions over the next week. Of course, most of these sessions are focussed on developers building apps for the Apple platforms. As a Mac and iOS admin, which sessions are interesting?
Of course, we won’t know which sessions are good until we can watch them. But here is the list of sessions that I expect to be interesting. I will update this post all week. Let me know if you find something interesting that I missed.
Apple releases the sessions every day of the WWC week at 9am PDT/17:00CEST. I will add the day of the week after the session title. You can watch the sessions in the Developer app on your Apple device and on Apple’s developer web page.
Keep in mind that while the excitement is huge around WWDC time, you do not have to watch all the sessions this week. The sessions will remain available and you can take your time to catch up.
The keynote is of course press- and end-user facing and very marketing driven. Still worth watching it (if you haven’t already) for the highlights. The ‘Platform State of the Union’ is the ‘real’ developer-focussed keynote. It is interesting to watch to understand where Apple thinks the focus is going to be. This year the highlights are the new Xcode Cloud, new Swift features, Object Capture and many new frameworks.
Device Management
There is actually a dedicated category for “Device Management” in the Developer app. Some of these sound very promising:
Some of the other sessions will ahve relevance for MacAdmins (and users) as well. Often these sessions will have segments with a general overview of a feature, followed by details on how to implement it in code.
When you can use your managed Apple ID from Apple Business Manager or Apple School Manager to log in to AppleSeed for IT. There you can download the beta systems to start testing now. AppleSeed for IT also contains more detailed release notes, which you find as a PDF under the downloads category.
Keep in mind that while you can discuss information released in the WWDC sessions in public, information that is exclusive to AppleSeed for IT and the other seed and beta programs is subject to the NDA, and should not be discussed in public forums.
This week Apple finally sent out the invitations for an event next week. The “Spring Loaded” event will take place next week, April 20, at 10am PDT.
Release of iOS 14.5, macOS 11.3, and siblings are likely on or around that day then. We did get an eighth beta this week. Next week should be interesting.
To support this weekly news summary, please consider:
macOS Terminal and Shell: You have always wanted to ‘learn Terminal,‘ right? This book teaches how (and why) to use the command line on macOS. Get it on Apple Books!
In other news, Parallels and Docker now have officially released Apple silicon native solutions. Ironically, Parallels cannot (yet?) host macOS Big Sur guest systems, only Windows 10 for ARM (preview) and ARM Linux systems.
The Docker does not come as a Universal download, but as two seperate downloads, which seems a common thing in this transition. Why is it that so many apps are separate downloads in this transition?
If you want to support me and this website even further, then consider buying one (or all) of my books. It’s like a subscription fee, but you also get a useful book or two extra!
In shell scripting, there are many situations where you want to know the path to the script file itself.
I use this frequently in scripts that build pkgs, because I want the script to access or create files relative to the script or its enclosing folder. In other words: the script sits in some project folder, and you want to get the path to that project folder.
Some scripts just use the current working directory, but since you can launch scripts from outside the current working directory, this is quite unreliable.
In that package build script I have the following line:
projectfolder=$(dirname "$0")
This is a useful but somewhat tricky line. It will work in most situations, but not all. To understand when it might fail, we will have to unravel it from the inside.
The $0 variable is the first element of the command line as it is entered in to the prompt and contains the command itself. In our case it will most likely be ./buildBoringDesktopPkg.sh.
The dirname command removes the last element from a path. This will return the path to the directory containing the item you give it. When we do dirname $0 after launching the script with ./buildBoringDesktopPkg.sh we will get . which is the current working directory, but also the parent directory of the script.
When you launch the build script with a different working directory, i.e. like
> Projects/BoringDesktop/buildBoringDesktopPkg.sh
The $0 argument will be Projects/BoringDesktop/buildBoringDesktopPkg.sh the dirname will return Projects/BoringDesktop/. This is all good and it will still find the resources and create the pkg file in its project folder.
In most situations the syntax used to call your script will be a relative or absolute path to the script. Something like ./buildPkg.sh or ~/Projects/Tools/my_script.sh. When the shell ‘sees’ a slash / in the command, it will assume this is the path to the script or executable and use that.
But when there is no slash / in the command, the shell will use the directories listed in the PATH environment variable to look for the command. This is the reason you can type sw_vers instead of having to write /usr/bin/sw_vers.
This is also the reason you have to type ./my_script.sh to run a script you are working on. Your script is not in the PATH, but the command contains a slash and tells the shell exactly where to look (in the current directory). When you just type my_script.sh the shell will look in the PATH directories, not find your script, and error out. (Unless your current directory happens to be in the PATH.)
This script shows the values of the current working directory, the $0 argument, and the value returned by dirname $0. Save this script as script_path_test.sh in your Desktop folder.
So far, so good. While some of these paths look a little odd, they all return the expected values.
But now imagine we like our script so much, that we use it a lot. To save on typing the path to the script all the time, we add a symbolic link to it to /usr/local/bin. This puts our script in the default PATH (for an interactive shell) and we can now execute it without needing to type the path to it:
So, this is less ideal. When you enter just script_path_test.sh the shell finds the symbolic link in /usr/local/bin and runs it. The $0 argument points to the symbolic link instead of the actual script and hence dirname $0 points to /usr/local/bin which is not where the resources for our script are.
(Remember to delete this symbolic link from /usr/local/bin/ when you are done with this.)
We need a tool that resolves symbolic links.
There are quite a few solutions to this, many of which can be found in this Stack Overflow post.
GNU-based systems have a realpath command line tool, which does exactly this. macOS does not have this tool. You can add GNU tools to macOS with many of the popular package managers, but that introduces a dependency and seems overkill.
Python has a function that does this. Add this line to the script:
But what if you have sh or bash scripts that you cannot just quickly convert to zsh? The good news here is that we can create a zsh one-liner that does this, even from a bash or sh script, very similar to the python one-liner above.
This weblog was (quite obviously) inspired by the weblogs of Greg Neagle, Rich Trouton, Ben Toms, and many others. I wanted to share and give back to the MacAdmins community. I have always enjoyed scripting, in the sense of tinkering and combining tools in useful, or sometimes just funny ways and sharing that seemed like an obvious thing to do.
I wasn’t really planning on honing my writing skills for writing books later. As many of these personal weblogs go, I posted quite irregularly for the first few years. Still, I was often quite excited when a post got dozens of views.
As it turns out, a weblog is also a decent marketing platform for a self-published book (or two, or three, or four). When you are writing a book, there are always a few things that don’t quite “fit in” and that then get turned into a blog post. Once I got into the habit of posting regularly, the viewer numbers and the books sales increased. At some point, my ranking in the weird black magic that is the Google algorithm reached some critical point and the views from searches started pouring in.
Having posts being read and re-shared does give you a bit of a thrill. And I wanted to share that feeling. I started the weekly news summary to have a place where all the great work of fellow MacAdmins (and related experts) is gathered. I got requests to have an email newsletter fairly quickly. We passed 1000 subscribers on the email newsletter in October, and many more read it on the website.
Last week, according to the Jetpack metrics, this site had its millionth unique visitor overall and its 500.000th unique visitor in this calendar year. Yes, that means traffic is more than doubling year over year.
Back when a post would get hundreds of views that was exciting. The traffic now is still exciting, but also a bit humbling. So, thank you all for being here and reading and sharing my posts and books.
There was never a big strategy or plan. But I am very happy how everything worked out. Over the years, writing for the weblog gave me confidence to write and self-publish a book. The books and posts lead to conference presentations, which lead to more posts and books and the newsletter. And the best part is: I got to meet and befriend some pretty great people from all over the world.
And true to the spirit of how the last ten years turned out, I have no plans or strategies for major changes in the near future. I will keep the things that work and try new things as they occur to me. That doesn’t mean that there won’t be a few minor changes happening soon, though.
I have pushed an update for the “Moving to zsh” book.
Big Sur is such an important update that gave it the long-awaited version number ’11.’ Thankfully, it did not bring many changes to the way the Terminal and zsh work.
zsh is still the (new) default shell for new users. bash v3 is (so far) still present on macOS Big Sur, but when you use it as your shell, you will get the warning to switch in Terminal. While Big Sur updates zsh to version 5.8 this doesn’t change any major behavior compared to zsh 5.7.1 in zsh.
Because of all this, I only had to do a few minor updates to the book.
I do anticipate that many user who have been holding off from upgrading to Catalina (or older versions of macOS), will now either upgrade to Catalina or leap frog directly to Big Sur. For those users, this book is ready to help them “Move to zsh.” Please, recommend this book when you encounter one of these users.
As usual, the update is free if you have already purchased the book. You should get a notification from the Books application to update. (On macOS, I have seen that it can help to delete the local download of the book to force the update. It might still take a few hours for the change to propagate through Apple’s server network. Even when you get the older version now, you can re-download the update when it is available.)
If you are enjoying the book, please rate it on the Books store, or even leave a review. These really help, thank you!
The changes are listed here. This list is also in the ‘Version History’ section in the book. There, you will get links to the relevant section of the book, so you can find the changes quickly.
Description of the new zsh session restore feature in Big Sur Terminal
Updated some images and text with macOS Big Sur information
Added a link to a post with instructions to install shellcheck on macOS
In one of the recent betas for macOS 11.0 Big Sur, the xpath command line tool changed. Big Sur uses the 5.28 version of the tool, while Catalina defaults to the 5.18 version.
These aren’t version numbers for xpath but actually version numbers for perl. When you look at all the xpath executables you will see that they are actually perl scripts.
The perl environment will choose the xpath script ending in the version number matching the perl version automatically. The plain xpath script with no version number serves as fallback.
Either way, Catalina runs perl 5.18 and xpath5.18 and Big Sur runs perl 5.28 and the newer xpath5.28 script.
The problem here is that the newer xpath script has a different syntax:
Either version will use stdin when there is no file name, but the newer xpath requires the query string to be labeled with a -e argument.
So, your scripts that are using the xpath tool to parse XML data, will fail in Big Sur!
The easiest fix is to change the script to use the new syntax, i.e. insert the -e at the right place and if necessary re-arrange the arguments. But then the script will fail on older versions of macOS. Many of us will have to write our scripts to be able to support the latest and older versions of macOS for a transition time. For some MacAdmins the transition time can be several years.
Since Big Sur still includes xpath5.18, another solution is to just hardcode the version whenever you use xpath. But this will only defer the problem to a future version of macOS, when the 5.18 version of the script is removed.
A better solution is to check the version of macOS that is being used and to call xpath with the proper for each version:
if [[ $(sw_vers -buildVersion) > "20A" ]]; then
xpath -e "//query" "/path/to/file"
else
xpath "/path/to/file" "//query"
fi
This will call xpath with the new syntax on Big Sur (20A) and higher and use the older syntax otherwise. (Why I use the build version.)
If you are using xpath in multiple locations in a script, using this code everywhere will become tedious. In the Installomator script, we often use xpath to parse the download URL out of an XML file. As of now, there are nine occurences of xpath, but as more people contribute to Installomator, this is likely to go up.
There is a nice sleight-of-hand trick you can use to make the script resilient to change in Big Sur, without having to touch every use of xpath in the code.
At the beginning of the script (before the first use of xpath) insert this function:
xpath() {
# the xpath tool changes in Big Sur
if [[ $(sw_vers -buildVersion) > "20A" ]]; then
/usr/bin/xpath -e "$@"
else
/usr/bin/xpath "$@"
fi
}
Since the shell interpreter (bash or zsh) will prefer a local function over an external executable, this function will now be used for all uses of xpath in the remaining code. We don’t need to touch them at all.
Within the function, we tell the interpreter to use the executable by using its full path.
In Installomator, we pipe data into xpath, so there are no files involved and the above works fine.
When you use xpath with files, it gets a bit more complicated, because the order of the arguments changed between the versions.
With the assumption that the current use is xpath [filename] query we can use the following:
xpath() { # [filename] query
# xpath in Big Sur changes syntax
if [[ -n $2 ]]; then
local query=$2
local filename=$1
else
local query=$1
local filename=""
fi
if [[ $(sw_vers -buildVersion) > "20A" ]]; then
/usr/bin/xpath -e "$query" "$filename"
else
/usr/bin/xpath "$filename" "$query"
fi
}
You will want to make a note for some point in the future, when you finally can drop Catalina support, to revisit these scripts and clean them up for the new syntax.
I have not yet encountered other command line tools that change in a similar way in Big Sur. If you do, you should be able to use a similar function to simplify the transtition.
by 
News and Opinion
Security and Privacy
Support and HowTos
Scripting and Automation
Updates and Releases
To Watch
To Listen
Just for Fun
Support