Weekly News Summary for Admins — 2021-04-16

This week Apple finally sent out the invitations for an event next week. The “Spring Loaded” event will take place next week, April 20, at 10am PDT.

Release of iOS 14.5, macOS 11.3, and siblings are likely on or around that day then. We did get an eighth beta this week. Next week should be interesting.


To support this weekly news summary, please consider:

macOS Terminal and Shell Book Cover

macOS Terminal and Shell:
You have always wanted to ‘learn Terminal,‘ right? This book teaches how (and why) to use the command line on macOS. Get it on Apple Books!

(If you have already bought the book, please leave a review on the Apple Books Store. Thank you!)


In other news, Parallels and Docker now have officially released Apple silicon native solutions. Ironically, Parallels cannot (yet?) host macOS Big Sur guest systems, only Windows 10 for ARM (preview) and ARM Linux systems.

The Docker does not come as a Universal download, but as two seperate downloads, which seems a common thing in this transition. Why is it that so many apps are separate downloads in this transition?

📰News and Opinion

🔐Security and Privacy

🔨Support and HowTos

🤖Scripting and Automation

♻️Updates and Releases

📺To Watch

🎧To Listen

🎈Just for Fun

📚 Support

If you are enjoying what you are reading here, please spread the word and recommend it to another Mac Admin!

If you want to support me and this website even further, then consider buying one (or all) of my books. It’s like a subscription fee, but you also get a useful book or two extra!

Script Identity Crisis – get the Path to the Current Script

In shell scripting, there are many situations where you want to know the path to the script file itself.

I use this frequently in scripts that build pkgs, because I want the script to access or create files relative to the script or its enclosing folder. In other words: the script sits in some project folder, and you want to get the path to that project folder.

Some scripts just use the current working directory, but since you can launch scripts from outside the current working directory, this is quite unreliable.

One example is this script from my book ‘Packaging for Apple Administrators‘ which builds an installer package for a desktop picture from resources in the same folder.

In that package build script I have the following line:

projectfolder=$(dirname "$0")

This is a useful but somewhat tricky line. It will work in most situations, but not all. To understand when it might fail, we will have to unravel it from the inside.

The $0 variable is the first element of the command line as it is entered in to the prompt and contains the command itself. In our case it will most likely be ./buildBoringDesktopPkg.sh.

The dirname command removes the last element from a path. This will return the path to the directory containing the item you give it. When we do dirname $0 after launching the script with ./buildBoringDesktopPkg.sh we will get . which is the current working directory, but also the parent directory of the script.

When you launch the build script with a different working directory, i.e. like

> Projects/BoringDesktop/buildBoringDesktopPkg.sh

The $0 argument will be Projects/BoringDesktop/buildBoringDesktopPkg.sh the dirname will return Projects/BoringDesktop/. This is all good and it will still find the resources and create the pkg file in its project folder.

In most situations the syntax used to call your script will be a relative or absolute path to the script. Something like ./buildPkg.sh or ~/Projects/Tools/my_script.sh. When the shell ‘sees’ a slash / in the command, it will assume this is the path to the script or executable and use that.

But when there is no slash / in the command, the shell will use the directories listed in the PATH environment variable to look for the command. This is the reason you can type sw_vers instead of having to write /usr/bin/sw_vers.

This is also the reason you have to type ./my_script.sh to run a script you are working on. Your script is not in the PATH, but the command contains a slash and tells the shell exactly where to look (in the current directory). When you just type my_script.sh the shell will look in the PATH directories, not find your script, and error out. (Unless your current directory happens to be in the PATH.)

Note: Learn the details about PATH in my new book: “macOS Terminal and Shell

So, there are a lot of pieces that are really confusing here. Let’s create a script that will help us visualize this:

#!/bin/sh

echo "pwd: $(pwd)"
echo "\$0: $0"
echo "dirname \$0: $(dirname $0)"

This script shows the values of the current working directory, the $0 argument, and the value returned by dirname $0. Save this script as script_path_test.sh in your Desktop folder.

> cd Desktop                      
> ./script_path_test.sh     
pwd: /Users/armin/Desktop
$0: ./script_path_test.sh
dirname $0: .
> cd ~                  
> Desktop/script_path_test.sh     
pwd: /Users/armin
$0: Desktop/script_path_test.sh
dirname $0: Desktop

So far, so good. While some of these paths look a little odd, they all return the expected values.

But now imagine we like our script so much, that we use it a lot. To save on typing the path to the script all the time, we add a symbolic link to it to /usr/local/bin. This puts our script in the default PATH (for an interactive shell) and we can now execute it without needing to type the path to it:

> sudo ln -s ~/Desktop/script_path_test.sh /usr/local/bin/script_path_test.sh
> script_path_test.sh
pwd: /Users/armin
$0: /usr/local/bin/script_path_test.sh
dirname $0: /usr/local/bin

So, this is less ideal. When you enter just script_path_test.sh the shell finds the symbolic link in /usr/local/bin and runs it. The $0 argument points to the symbolic link instead of the actual script and hence dirname $0 points to /usr/local/bin which is not where the resources for our script are.

(Remember to delete this symbolic link from /usr/local/bin/ when you are done with this.)

We need a tool that resolves symbolic links.

There are quite a few solutions to this, many of which can be found in this Stack Overflow post.

GNU-based systems have a realpath command line tool, which does exactly this. macOS does not have this tool. You can add GNU tools to macOS with many of the popular package managers, but that introduces a dependency and seems overkill.

Python has a function that does this. Add this line to the script:

echo "python realpath: $(python -c "import os; print(os.path.realpath('$0'))")"

This works fine, however, the built-in Python for macOS is deprecated. We don’t want to introduce this dependency, when it might go away in a future macOS update.

Then I found this post where I learned that zsh has a parameter expansion modifier to return the absolute path while resolving symbolic links.

Let’s modify our script to use zsh:

#!/bin/zsh

echo "pwd: $(pwd)"
echo "\$0: $0"
echo "dirname \$0: $(dirname $0)"

echo "\${0:A}: ${0:A}"
echo "dirname \${0:A}: $(dirname ${0:A})"

and then we can run our script:

> script_path_test.sh
pwd: /Users/armin
$0: /usr/local/bin/script_path_test.sh
dirname $0: /usr/local/bin
${0:A}: /Users/armin/Desktop/script_path_test.sh
dirname ${0:A}: /Users/armin/Desktop

Zsh to the rescue. We have a solution! Zsh is pre-installed on every macOS and since Apple chose it as their default shell since Catalina, it will be around for a while longer.

But what if you have sh or bash scripts that you cannot just quickly convert to zsh? The good news here is that we can create a zsh one-liner that does this, even from a bash or sh script, very similar to the python one-liner above.

echo "zsh absolute path: $(zsh -c 'echo ${0:A}' "$0")"
echo "zsh dirname absolute: $(dirname $(zsh -c 'echo ${0:A}' "$0"))"

So, in conclusion:

  • dirname $0 will give you the enclosing folder of the script in most situations
  • with zsh you can just use dirname ${0:A} to be even safer
  • when you have to use sh or bash, you can use a zsh one-liner: dirname $(zsh -c 'echo ${0:A}' "$0").

Happy Scripting!

(Remember to delete the symbolic link from /usr/local/bin/!)

Ten Years of Scripting OS X

The oldest post on this weblog happened ten years ago. It makes me especially proud, that the first post has aged quite well. (The second post, not so much.)

This weblog was (quite obviously) inspired by the weblogs of Greg Neagle, Rich Trouton, Ben Toms, and many others. I wanted to share and give back to the MacAdmins community. I have always enjoyed scripting, in the sense of tinkering and combining tools in useful, or sometimes just funny ways and sharing that seemed like an obvious thing to do.

I wasn’t really planning on honing my writing skills for writing books later. As many of these personal weblogs go, I posted quite irregularly for the first few years. Still, I was often quite excited when a post got dozens of views.

As it turns out, a weblog is also a decent marketing platform for a self-published book (or two, or three, or four). When you are writing a book, there are always a few things that don’t quite “fit in” and that then get turned into a blog post. Once I got into the habit of posting regularly, the viewer numbers and the books sales increased. At some point, my ranking in the weird black magic that is the Google algorithm reached some critical point and the views from searches started pouring in.

Having posts being read and re-shared does give you a bit of a thrill. And I wanted to share that feeling. I started the weekly news summary to have a place where all the great work of fellow MacAdmins (and related experts) is gathered. I got requests to have an email newsletter fairly quickly. We passed 1000 subscribers on the email newsletter in October, and many more read it on the website.

Last week, according to the Jetpack metrics, this site had its millionth unique visitor overall and its 500.000th unique visitor in this calendar year. Yes, that means traffic is more than doubling year over year.

Back when a post would get hundreds of views that was exciting. The traffic now is still exciting, but also a bit humbling. So, thank you all for being here and reading and sharing my posts and books.

There was never a big strategy or plan. But I am very happy how everything worked out. Over the years, writing for the weblog gave me confidence to write and self-publish a book. The books and posts lead to conference presentations, which lead to more posts and books and the newsletter. And the best part is: I got to meet and befriend some pretty great people from all over the world.

And true to the spirit of how the last ten years turned out, I have no plans or strategies for major changes in the near future. I will keep the things that work and try new things as they occur to me. That doesn’t mean that there won’t be a few minor changes happening soon, though.

On to the next ten years!

Book Update for Big Sur – Moving to zsh v4

I have pushed an update for the “Moving to zsh” book.

Big Sur is such an important update that gave it the long-awaited version number ’11.’ Thankfully, it did not bring many changes to the way the Terminal and zsh work.

zsh is still the (new) default shell for new users. bash v3 is (so far) still present on macOS Big Sur, but when you use it as your shell, you will get the warning to switch in Terminal. While Big Sur updates zsh to version 5.8 this doesn’t change any major behavior compared to zsh 5.7.1 in zsh.

Because of all this, I only had to do a few minor updates to the book.

I do anticipate that many user who have been holding off from upgrading to Catalina (or older versions of macOS), will now either upgrade to Catalina or leap frog directly to Big Sur. For those users, this book is ready to help them “Move to zsh.” Please, recommend this book when you encounter one of these users.

As usual, the update is free if you have already purchased the book. You should get a notification from the Books application to update. (On macOS, I have seen that it can help to delete the local download of the book to force the update. It might still take a few hours for the change to propagate through Apple’s server network. Even when you get the older version now, you can re-download the update when it is available.)

When you haven’t gotten the book yet, you can purchase it on the Apple Books store.

If you are enjoying the book, please rate it on the Books store, or even leave a review. These really help, thank you!

The changes are listed here. This list is also in the ‘Version History’ section in the book. There, you will get links to the relevant section of the book, so you can find the changes quickly.

  • Description of the new zsh session restore feature in Big Sur Terminal
  • Updated some images and text with macOS Big Sur information
  • Added a link to a post with instructions to install shellcheck on macOS

Dealing with xpath changes in Big Sur

In one of the recent betas for macOS 11.0 Big Sur, the xpath command line tool changed. Big Sur uses the 5.28 version of the tool, while Catalina defaults to the 5.18 version.

These aren’t version numbers for xpath but actually version numbers for perl. When you look at all the xpath executables you will see that they are actually perl scripts.

> head -n1 /usr/bin/xpath*
==> /usr/bin/xpath <==
#!/usr/bin/perl

==> /usr/bin/xpath5.18 <==
#!/usr/bin/perl5.18 -w

==> /usr/bin/xpath5.28 <==
#!/usr/bin/perl5.28

The perl environment will choose the xpath script ending in the version number matching the perl version automatically. The plain xpath script with no version number serves as fallback.

Either way, Catalina runs perl 5.18 and xpath5.18 and Big Sur runs perl 5.28 and the newer xpath5.28 script.

The problem here is that the newer xpath script has a different syntax:

[5.18] xpath [filename] query
[5.28] xpath [options] -e query [-e query...] [filename...]

Either version will use stdin when there is no file name, but the newer xpath requires the query string to be labeled with a -e argument.

So, your scripts that are using the xpath tool to parse XML data, will fail in Big Sur!

The easiest fix is to change the script to use the new syntax, i.e. insert the -e at the right place and if necessary re-arrange the arguments. But then the script will fail on older versions of macOS. Many of us will have to write our scripts to be able to support the latest and older versions of macOS for a transition time. For some MacAdmins the transition time can be several years.

Since Big Sur still includes xpath5.18, another solution is to just hardcode the version whenever you use xpath. But this will only defer the problem to a future version of macOS, when the 5.18 version of the script is removed.

A better solution is to check the version of macOS that is being used and to call xpath with the proper for each version:

    if [[ $(sw_vers -buildVersion) > "20A" ]]; then
        xpath -e "//query" "/path/to/file"
    else
        xpath "/path/to/file" "//query" 
    fi

This will call xpath with the new syntax on Big Sur (20A) and higher and use the older syntax otherwise. (Why I use the build version.)

If you are using xpath in multiple locations in a script, using this code everywhere will become tedious. In the Installomator script, we often use xpath to parse the download URL out of an XML file. As of now, there are nine occurences of xpath, but as more people contribute to Installomator, this is likely to go up.

There is a nice sleight-of-hand trick you can use to make the script resilient to change in Big Sur, without having to touch every use of xpath in the code.

At the beginning of the script (before the first use of xpath) insert this function:

xpath() {
    # the xpath tool changes in Big Sur 
    if [[ $(sw_vers -buildVersion) > "20A" ]]; then
        /usr/bin/xpath -e "$@"
    else
        /usr/bin/xpath "$@"
    fi
}

Since the shell interpreter (bash or zsh) will prefer a local function over an external executable, this function will now be used for all uses of xpath in the remaining code. We don’t need to touch them at all.

Within the function, we tell the interpreter to use the executable by using its full path.

In Installomator, we pipe data into xpath, so there are no files involved and the above works fine.

When you use xpath with files, it gets a bit more complicated, because the order of the arguments changed between the versions.

With the assumption that the current use is xpath [filename] query we can use the following:

xpath() { # [filename] query
    # xpath in Big Sur changes syntax
    if [[ -n $2 ]]; then
        local query=$2
        local filename=$1
    else 
        local query=$1
        local filename=""
    fi

    if [[ $(sw_vers -buildVersion) > "20A" ]]; then
        /usr/bin/xpath -e "$query" "$filename"
    else
        /usr/bin/xpath "$filename" "$query"
    fi
}

You will want to make a note for some point in the future, when you finally can drop Catalina support, to revisit these scripts and clean them up for the new syntax.

I have not yet encountered other command line tools that change in a similar way in Big Sur. If you do, you should be able to use a similar function to simplify the transtition.

When you do, let us know in the comments!

Installomator Updated: v0.4

I had been fairly busy with both the JNUC and MacSysAdmin presentations, but work on Installomator has continued.

I also did an overview of what Installomator does in my MacSysAdmin Online presentation: “Practical Scripting”.

Many of these new features have been provided from others, either through GitHub issues, pull requests, or through comments in the #installomator channel on MacAdmins Slack. Thanks to all who contributed.

What’s new in v0.4:

  • you can now set script variables as an argument in the form VARIABLE=value. More detail on this in the README file, ‘Configuration from Arguments.’
  • change downloadFromGit to match file types better
  • implemented a workaround for changed behavior of xpath in Big Sur
  • added an option prompt_user_the_kill to BLOCKING_PROCESS_ACTION which will kill the process after the third unsuccessful attempt to quit
  • added several new labels for total of 116

Get the script and find the instructions on the GitHub repo.

If you have any feedback or questions, please join us in the #installomatorchannel on MacAdmins Slack.

Thanks again to all those who contributed!

Weekly News Summary for Admins — 2020-10-16

We got the expected new iPhones 12 this week, all four models, and a new HomePod mini. I am actually looking forward to a smaller iPhone 12 mini to replace my iPhone X.

In other (some would say good) news, macOS 11 Big Sur was not released this week, but we got beta 10 instead. There is an expectation for another Apple event introducing the first Apple Silicon based Macs in the upcoming weeks and that would be an obvious time to release Big Sur to everyone.

If you would rather get the weekly newsletter by email, you can subscribe to the Scripting OS X Weekly Newsletter here!! (Same content, delivered to your Inbox once a week.)

News and Opinion

macOS 11 Big Sur, Apple Silicon, and iOS 14

MacAdmins on Twitter

  • Taha Karim: “MacOS Ransomware in one tweet” (Thread, via Mat X)
  • William Smith: “Tomorrow (October 13, 2020): Support for Microsoft Office 2016 for Mac — any version 16.16.x and lower— ends. No more software updates to fix: • Bugs • Compatibility problems • Security vulnerabilities – No macOS Big Sur support”

Bugs and Security

Support and HowTos

Scripting and Automation

Updates and Releases

To Watch

To Listen

Support

If you are enjoying what you are reading here, please spread the word and recommend it to another Mac Admin!

If you want to support me and this website even further, then consider buying one (or all) of my books. It’s like a subscription fee, but you also get a useful book or two extra!

Weekly News Summary for Admins — 2020-10-09

For now this has been a quiet week. Since Apple announced another event next week, possibly a calm before the storm. After Apple announced iOS 14 with 24 hours notice, MacAdmins are understandably a bit nervous. Also, no new betas this week (so far).

On the other hand, after JNUC last week, we had MacSysAdmin this week. I have not been able to watch all presentations yet, but those that I have seen have been worth it and I am looking forward to the rest.

In other news, this newsletter just passed the 1000 email subscribers number! Thank you all for reading, and on to the next thousand!

If you would rather get the weekly newsletter by email, you can subscribe to the Scripting OS X Weekly Newsletter here!! (Same content, delivered to your Inbox once a week.)

News and Opinion

macOS 11 Big Sur and iOS 14

MacAdmins on Twitter

  • Eliz: “Please don’t say just “Hello” in a chat #NoHello” (Read thread and links for background. I find myself doing that a lot, too, and have resvoled to do better.)

Bugs and Security

Support and HowTos

Scripting and Automation

Updates and Releases

To Watch

To Listen

Support

If you are enjoying what you are reading here, please spread the word and recommend it to another Mac Admin!

If you want to support me and this website even further, then consider buying one (or all) of my books. It’s like a subscription fee, but you also get a useful book or two extra!

Installomator Updated: v0.3

It’s been more than a month since the last update, and while there has been work on the dev branch, I was quite distracted with other things (like this). The good news is, that there have been quite a few contributions from others! A huge thanks to all who helped make this a better script.

All it took was for me to find some time to put all the contributions together, which I finally found some time for.

What’s new in v0.3:

  • added several new labels for total of 98
  • removed the powershell labels, since the installer is not notarized
  • when run without any arguments, the script now lists all labels
  • changed how zips are expanded because this was broken on Mojave
  • improved logging in some statements
  • several more minor improvements

Get the script and find the instructions on the GitHub repo.

Some of the contributions and requests have not yet been addressed. I believe they will require some more thinking and planning. I would like to approach those in the next version.

If you have any feedback or questions, please join us in the #installomator channel on MacAdmins Slack.

Thanks again to all those who contributed!

Installomator updated: v0.2

It’s been nearly a month since I introduced Installomator.

Since then, it has gotten lots of feedback from others and many contributions. As the changes, fixes and additional apps have accumulated, I have created a 0.2 release to get a stable new version. If you like living on the edge you can also use the dev branch for the latest update.

Changes in this version:

  • many fixes for broken URLs and other bugs
  • pkgInDmg and pkgInZip now search for the first pkg file in the archive in case the file name varies with the version
  • notification on successful installation can be suppressed with the NOTIFY variable
  • Apple signed installers and apps that don’t have a Team ID are verified correctly now
    improved logging
  • several new applications: count increased from 62 in v0.1 to 87 in v0.2

Thanks to all who contributed!

Also, if you haven’t already, you want to read Mischa’s guest post on using Installomator with Jamf Pro.