The unexpected return of JavaScript for Automation

Monterey has deprecated the pre-installed python on macOS. To be precise, built-in python has been deprecated since macOS Catalina, but Monterey will now throw up dialogs warning the user that an app or process using built-in python needs to be updated.

I and others have written about this before:

• Monterey, python, and free disk space – Scripting OS X
• macOS Monterey prompt: “…..” needs to be updated – Ben Toms
• MDM custom preference to disable python 2 deprecation popups – Graham R Pugh
• Wrangling Pythons – Scripting OS X
• Get Current User in Shell Scripts on macOS – Scripting OS X

So far, I have recommended to build native Swift command line tools to replace python calls. However, from discussions in MacAdmins Slack, a new option has emerged. Most of the credit for popularizing and explaining this goes to @Pico (@RandomApps on Twitter) in the #bash and #scripting channels.

(Re-)Introducing JavaScript for Automation

AppleScript has been part of macOS since System 7.1. In the late nineties, there was concern that it wouldn’t make the transition to Mac OS X, but AppleScript made the jump and has happily co-existed with the Terminal and shell scripting as an automation tool on macOS. AppleScript has a very distinct set of strengths (interapplication communication) and weaknesses (awkward syntax and inconsitent application functionality and dictionaries) but it has been serving its purpose well for many users.

With Mac OS X 10.4 Tiger, Apple introduced Automator, which provided a neat UI to put together workflows. Much of Automator was based on AppleScript and users expected a more and improved AppleScript support because of that going forward. Instead, we saw AppleScript’s support from Apple and third parties slowly wane over the years.

AppleScript is stil very much present and functional in recent versions of macOS. It just seems like it hasn’t gotten much love over the last decade or so. Now that Shortcuts has made the jump from iOS, there may be hope for another revival?

The last major changes to AppleScript came with Mavericks and Yosemite. Mavericks (10.9) included a JavaScript syntax for the Open Scripting Architecture (OSA), which is the underlying framework for all AppleScript functionality. Apple called this “JavaScript for Automation.” Because this is a mouthful, it often abbreviated as JXA.

The JavaScript syntax and structure is more like a “real” programming language, than the “english language like” AppleScript. Once again this raised hopes that this could attract more scripters to AppleScript and thus encourage Apple and third party developers to support more AppleScript. But unfortunately, this positive re-inforcement did not take off.

Then Yosemite (10.10) made the AppleScript-Objective-C bridge available everywhere in AppleScript. Previously, the Objective-C bridge was only available when you built AppleScript GUI applications using AppleScript Studio in Xcode. The Objective-C bridge allows scripters to access most of the functionality of the system frameworks using AppleScript or JXA.

The coincidence of these two new features might be the reason that the ObjC bridge works much better using JXA than it does with the native AppleScript syntax.

JXA and Python

What does JXA and the AppleScriptObjC bridge have to do with the Python deprecation in modern macOS?

One reason python became so popular with MacAdmins, was that the pre-installed python on Mac OS X, also came with PyObjC, the Objective-C bridge for python. This allowed python to build applications with a native Cocoa UI, such as AutoDMG and Munki’s Managed Software Center. It also allowed for short python scripts or even one-liners to access system functionality that was otherwise unavailable to shell scripts.

For example, to determine if a preference setting in macOS is enforced with a configuration profile, you can use CFPreferences or NSUserDefaults.

Objective-C/C:

BOOL isManaged =CFPreferencesAppValueIsForced("idleTime", "com.apple.screensaver")

Swift:

let isManaged = CFPreferencesAppValueIsForced("idleTime", "com.apple.screensaver")

The Objective-C bridge allows to use this call from python, as well:

from Foundation import CFPreferencesAppValueIsForced
isManaged=CFPreferencesAppValueIsForced("idleTime", "com.apple.screensaver")

With JXA and the AppleScriptObjC bridge, this will look like this:

ObjC.import('Foundation');
$.CFPreferencesAppValueIsForced(ObjC.wrap('idleTime'), ObjC.wrap('com.apple.screensaver'))

Now, this looks really simple, but working with any Objective-C bridge is always fraught with strange behaviors, inconsistencies and errors and the JXA ObjC implementation is no different.

For example, I wanted to change the code above to return the value of the setting instead of whether it is managed. The CFPreferences function for that is called CFPreferencesCopyAppValue and it works fine in Swift and Python, but using JXA it only ever returned [object Ref]. The easiest solution was to switch from the CFPreferences functions to using the NSUserDefaults object:

ObjC.import('Foundation');
ObjC.unwrap($.NSUserDefaults.alloc.initWithSuiteName('$1').objectForKey('$2'))

(Once again many thanks to @Pico on the MacAdmins Slack for helping me and everyone else with this and also pointing out, that there is a different, somewhat complicated, solution to the object Ref problem. I will keep that one bookmarked for situations where there is no alternative Cocoa object.)

We used this to remove the python dependency from Mischa van der Bent’s CIS-Scripts.

JXA in shell scripts

To call JXA from a shell script, you use the same osascript command as for normal AppleScript, but add the -l option option to switch the language to JavaScript:

osascript -l JavaScript << EndOfScript
     ObjC.import('Foundation');
    ObjC.unwrap($.NSUserDefaults.alloc.initWithSuiteName('idleTime').objectForKey('com.apple.screensaver'))
EndOfScript

For convenience, you can wrap calls like this in a shell function:

function getPrefValue() { # $1: domain, $2: key
      osascript -l JavaScript << EndOfScript
     ObjC.import('Foundation');
    ObjC.unwrap($.NSUserDefaults.alloc.initWithSuiteName('$1').objectForKey('$2'))
EndOfScript
}

function getPrefIsManaged() { # $1: domain, $2: key
     osascript -l JavaScript << EndOfScript
     ObjC.import('Foundation')
     $.CFPreferencesAppValueIsForced(ObjC.wrap('$1'), ObjC.wrap('$2'))
EndOfScript
}

echo $(getPrefValue "com.apple.screensaver" "idleTime")
# -> actual value
echo $(getPrefIsManaged "com.apple.screensaver" "idleTime")
# -> true/false

Note that the $ character does a lot of work here. It does the shell variable substitution for the function arguments in the case of $1 and $2. These are substituted before the here doc is piped into the osascript command. The $. at the beginning of the command is a shortcut where $ stands in for the current application and serves as a root for all ObjC objects.

There is also a $(…) function in JXA which is short for ObjC.unwrap(…) but I would recommend against using that in combination with shell scripts as shell’s command substitution has the same syntax and would happen before the JavaScript is piped into osascript.

There is a GitHub wiki with more detailed documentation on using JXA, and the JXA Objective-C bridge in particular.

JXA for management tasks

I’ll be honest here and admit that working with JXA seems strange, inconsistent, and — in weird way — like a step backwards. Putting together a Command Line Tool written in Swift feels like a much more solid (for lack of a better word) way of solving a problem.

However, the Swift binary command line tool has one huge downside: you have to install the binary on the client before you can use it in scripts and your management system. Now, as MacAdmins, we usually have all the tools and workflows available to install and manage software on the client. That’s what we do.

On the other hand, I have encountered three situations (set default browser, get free disk space, determine if a preference is managed) where I needed to replace some python code in the last few months and I would have no trouble finding a few more if I thought about it. Building, maintaining, and deploying a Swift CLI tool for each of these small tasks would add up to a lot of extra effort, both for me as the developer and any MacAdmin who wants to use the tools.

Alternatively, you can deploy and use a Python 3 runtime with PyObjC, like the MacAdmins Python and continue to use python scripts. That is a valid solution, especially when you use other tools built in python, like Outset or docklib. But it still adds a dependency that you have to install and maintain.

In addition to being extra work, it adds some burden to sharing your solutions with other MacAdmins. You can’t just simply say “here’s a script I use,” but you have to add “it depends on this runtime or tool, which you also have to install.

Dependencies add friction.

This is where JXA has an advantage. Since AppleScript and its Objective-C bridge are present on every Mac (and have been since 2014 when 10.10 was released) there is no extra tool to install and manage. You can “just share” scripts you build this way, and they will work on any Mac.

For example, I recently built a Swift command line tool to determine the free disk space. You can download the pkg, upload it to your management system, deploy it on your clients and then use a script or extension attribute or fact or something like to report this value to your management system. Since there is a possibility that the command line tool is not yet installed when the script runs, you need to add some code to check for that. All-in-all, nothing here is terribly difficult or even a lot of work, but it adds up.

Instead you can use this script (sample code for a Jamf extension attribute):

#!/bin/sh

freespace=$(/usr/bin/osascript -l JavaScript << EndOfScript
    ObjC.import('Foundation')
    var freeSpaceBytesRef=Ref()
    $.NSURL.fileURLWithPath('/').getResourceValueForKeyError(freeSpaceBytesRef, 'NSURLVolumeAvailableCapacityForImportantUsageKey', null)
    ObjC.unwrap(freeSpaceBytesRef[0])
EndOfScript
)

echo "<result>${freespace}</result>"

Just take this and copy/paste it in the field for a Jamf Extension Attribute script and you will get the same same free disk space value as the Finder does. If you are running a different management solution, it shouldn’t be too difficult to adapt this script to work there.

The Swift tool is nice. Once it is deployed, there are some use cases where it could be useful to have a CLI tool available. But most of the time, the JXA code snippet will “do the job” with much less effort.

Note on Swift scripts

Some people will interject with “but you can write scripts with a swift shebang!” And they are correct. However, scripts with a swift shebang will not run on any Mac. They will only run with Xcode, or at least the Developer Command Line Tools, installed. And yes, I understand this is hard for developers to wrap their brains around, but most people don’t have or need Xcode installed.

When neither of these are installed yet, and your management system attempts to run a script with a swift shebang, it will prompt the user to install the Developer command line tools. This is obviously not a good user experience for a managed deployment.

As dependencies go, Xcode is a fairly gigantic installation. The Developer Command Line Tools much less so, but we are back in the realm of “install and manage a dependency.”

Parsing JSON

Another area where JXA is (not surprisingly) extremely useful is JSON parsing. There are no built-in tools in macOS for this so MacAdmins either have to install jq or scout or fall back to parsing the text with sed or awk. Since JSON is native JavaScript, JXA “just works” with it.

For example the new networkQuality command line tool in Monterey has a -c option which returns JSON data instead of printing a table to the screen. In a shell script, we can capture the JSON in a variable and substitute it into a JXA script:

#!/bin/sh

json=$(networkQuality -c)

osascript -l JavaScript << EndOfScript
    var result=$json
    console.log("Download:  " + result.dl_throughput)
    console.log("Upload:    " + result.ul_throughput)
EndOfScript

Update: (2021-11-24) Paul Galow points out that this syntax might allow someone to inject code into my JavaScript. This would be especially problematic with MacAdmin scripts as those often run with root privileges. The way to avoid this injection is too parse the JSON data with JSON.parse :

#!/bin/sh 

json=$(networkQuality -c) 

osascript -l JavaScript << EndOfScript     
  var result=JSON.parse(\`$json\`)     
  console.log("Download:  " + result.dl_throughput)     
  console.log("Upload:    " + result.ul_throughput) 
EndOfScript

(I am leaving the original code up there for comparison.)

Conclusion

After being overlooked for years, JXA now became noticeable again as a useful tool to replace python in MacAdmin scripts, without adding new dependencies. The syntax and implementation is inconsistent, buggy, and frustrating, but the same can be said about the PyObjC bridge, we are just used it. The community knowledge around the PyObjC bridge and solutions goes deeper.

However, as flawed as it is, JXA can be a simple replacement for the classic python “one-liners” to get data out of a macOS system framework. Other interesting use cases are being discovered, such as JSON parsing. As such, JavaScript for Automation or JXA should be part of a MacAdmins tool chest.