Installomator v10.6

It is that time again, we have released a new version of Installomator!

Many thanks to all the maintainers and contributors!

It has been quite a while since we published a release and there was quite a backlog. In some ways, Installomator is a victim of its own success. It is quite easy to create a new label and many contributors are doing that. It is also easy to provide an update to a broken label and while many contributors are doing that, there are fewer of them.

This means that we are accumulating a lot of stale and broken labels. Since there is now way for us to gather information on which labels are actually being used and how much, we end up with lot of turnover.

The team of maintainers has some ideas to tackle some of these problems with automation, but human curation will have to remain a step all along the way.

You can get the latest version of Installomator on the repo or download the installer pkg from here.

Building a LaunchD Installer pkg for desktoppr (and other tools)

In my last post introducing desktoppr 0.5, I said that a Mac admin could build a customized pkg with a customized LaunchAgent property list, that would run desktoppr manage at login and every few hours in the background to re-set the wallpaper when the configuration profile is updated.

I admit I glossed over the process of building a package that installs and launches a LaunchAgent (or LaunchDaemon). Since that is not a trivial configuration, I will describe the details in this post.

LaunchAgent? LaunchDaemon? LaunchD?

On macOS, launchd is the system-wide process that controls and launches every other process running on the system. When you look at the hierarchical process list in Activity Monitor, you will always see launchd running with the process ID 1 as the single child process of kernel_task and all other processes are a child of launchd.

More practically, users and admins can use launchd to control processes that need to be available at all times or be launched in certain intervals or at certain events.

launchd has two types of processes: LaunchDaemons and LaunchAgents. Since the terms daemons (or demons) and agents do not have clear definitions outside of launchd, a bit of explanation is required.

LaunchDaemons

LaunchDaemons are processes that run with system privileges (“as root”). Processes that belong to your management system and security suites may need to run periodically or on certain events in the background and require root level access to the system to do their jobs. Other processes or services need to listen for incoming network traffic on privileged ports.

Note that background process are often called ‘agents’ regardless of whether they run with root privileges or as a user. The launchd terminology is more precise here.

Apple has done a lot of work over the past few year to provide Endpoint Security or Network Extension frameworks that mitigate the need for third party root daemons, but there are still relevant use cases, especially when you are a managing Macs.

A LaunchDaemon is configured by providing a property list file in /Library/LaunchDaemons. This configuration plist will be automatically loaded at reboot, or can be loaded using the launchctl command.

LaunchAgents

LaunchAgents are processes that run with user privileges. Generally, if something needs to read or write to anything in the user home folder, it should be a LaunchAgent. Over the recent years, Apple has put restrictions on what data process can access in user space, which has made LaunchAgents more of a pain to manage, but they still have their use cases.

A LaunchAgent is configured by providing a property list file in /Library/LaunchAgents on the system level or in ~/Library/LaunchAgents in a user’s home folder. Configuration property lists that are in a user home folder, will only run for that user, while LaunchAgents that are configured in the system wide /Library/LaunchAgents will be loaded for all users on the system.

A LaunchAgent configuration plist will be automatically loaded at login, or can be loaded using the launchctl command.

Since desktoppr affects a user setting (the desktop/wallpaper), it has to run as the user, so we need to build and install a LaunchAgent. Creating and installing a LaunchDaemon is pretty much the same process, though.

Ingredients

To build an installation package to install and run a LaunchAgent or LaunchDaemon, you need three ingredients:

  • the binary or script which will be launched
  • a launchd configuration property list
  • installation scripts to unload and load the configuration

For desktoppr, we already have the binary (downloadable as a zip from the project release page) and a sample configuration property list file.

The sample launchd configuration looks like this:

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
    <key>Label</key>
    <string>com.scriptingosx.desktopprmanage</string>
    <key>ProgramArguments</key>
    <array>
        <string>/usr/local/bin/desktoppr</string>
        <string>manage</string>
    </array>
    <key>RunAtLoad</key>
    <true/>
    <key>StartInterval</key>
    <integer>10800</integer>
</dict>
</plist>

The label is required and is used to uniquely identify the LaunchAgent or Daemon with launchd and the launchctl command. Reverse domain notification is common. The filename of the configuration property list should match the label.

The ProgramArguments key provides an array with the command and arguments that will be launched. The first item in the array should provide the full path to the binary or script. Then you provide arguments, one string item per argument. Since the individual arguments are separated by the string tags, you do not need to escape spaces or other special characters in the arguments.

It is worth noting that launchd will launch the binary or script directly, so syntax and substitutions that are done in shells will not work. This includes variable and command substitution like $HOME, $USER, or $(date) as well as pipes and output redirection. You get a single command with a list of static arguments. (You can use the StandardOutPath and StandardErrorPath keys in the launchd config plist to redirect output.)

For the desktoppr LaunchAgent, that suits us just fine, as we are only passing the manage argument which tells desktoppr to get the details from a configuration profile (sample here).

The next key, RunAtLoad tells launchd, to run the binary with the arguments immediately when the configuration file is loaded. Since configuration files in /Library/LaunchAgents and ~/Library/LaunchAgents are automatically loaded when a user logs in, this generally means desktoppr manage will be run when a user logs in, which suits us well.

The last key StartInterval tells launchd to re-launch the process after a certain time (given in seconds, the 10800 seconds in our sample file translate to three hours). Should the system be sleeping at the time, it will not run the LaunchAgent at that time, or when the system wakes up, but wait until the next interval period comes around.

There are other keys that control when the process gets launched, such as StartCalendarInterval or WatchPath. You can read details in the man page for launchd.plist or on the excellent launchd.info page. There are also apps, like Peter Borg’s Lingon that provide a user interface for creating these plist files.

Move or copy the plist file into the correct folder. The plist files in /Library/LaunchDaemons and /Library/LaunchAgents must be owned by root. They must not be writable by group or other. The file mode 644 (rw-r--r--) is recommended. Similarly, the binary or script needs to be owned by root and cannot be writable by group or other. The file mode 755 (rwxr-xr-x) is recommended. Wrong file privileges will result in a Load/Bootstrap failed: 5: Input/output error.

Make sure that the binary or script does not have a quarantine flag attached. You can check with ls -al@ or xattr -p com.apple.quarantine /path/to/file and remove a quarantine flag with xattr -d com.apple.quarantine /path/to/file. When the binary or script has the quarantine flag, the configuration file will load fine but the actual execution will fail quietly.

Loading the LaunchAgent or LaunchDaemon

The easiest way to load a LaunchDaemon is to restart the Mac. The easiest way to load a LaunchAgent is to log out and login. This is usually not practical.

The launchctl command manages LaunchDaemons and LaunchAgents and we can use this to load LaunchAgents and LaunchDaemons on demand.

To load our desktoppr agent, use

> launchctl load /Library/LaunchAgents/com.scriptingosx.desktopprmanage.plist

People are already readying their pitchforks here. “Hold on,” they say. “Apple has marked load and unload as ‘legacy.’ You shouldn’t use them!”

While it is true that load and unload (and a bunch of other commands) are labeled as ‘legacy’ in the launchctl man page, this does not mean they are deprecated and should be avoided. The difference between the legacy and the ‘new’ commands is that the legacy commands pick up whether a process should be run in user or root context (the ‘target domain’) from the context that launchctl runs in, whereas the new bootstrap and bootout commands need the target domain to be stated explicitly.

This makes the modern commands more precise, but often more wordy to use. You will see that the legacy commands are simpler to use in the interactive terminal, while the ‘modern’ commands are more useful in scripts. The equivalent ‘modern’ command is:

> launchctl bootstrap gui/501 /Library/LaunchAgents/com.scriptingosx.desktopprmanage.plist

where 501 is the user ID for your user account. This is 501 by default for the first user on on a macOS system, but you should verify what your account’s user ID actually is with id -u.

See what I mean with “more wordy to use”? That doesn’t mean they are bad or should be avoided, but that we can be picky about when to use which format. You could argue that using the modern commands throughout would be more consistent, but you still find the ‘legacy’ command is lots of documentation, so I feel a Mac admin needs to be aware of both.

Note: the ‘modern’ commands were introduced in OS X Yosemite 10.10 in 2014.

To load a LaunchDaemon, you need to run launchctl with root privileges. In the interactive shell, that means with sudo:

> sudo launchctl load /Library/LaunchDaemons/com.example.daemon.plist

or

> sudo launchctl bootstrap system/ /Library/LaunchDaemons/com.example.daemon.plist

To stop a LaunchAgent from being launched going forward, unload the configuration:

> launchctl unload /Library/LaunchAgents/com.scriptingosx.desktopprmanage.plist

or

> launchctl bootout gui/501/com.scriptingosx.desktopprmanage

The bootout command uses the label instead of the file path.

Putting it all together in an installer

Note: I will be showing how to build and installer package for our LaunchAgent using command line tools. These instructions should have all the information you need, even if you prefer using apps such as Whitebox Packages or Composer.

If you are not comfortable with using Terminal on macOS yet, please consider my book “macOS Terminal and Shell.” If you want to learn more about building installation packages, consider my book “Packaging for Apple Administrators.”

Download the zip file for desktoppr from the releases page. Download the sample launchd property list file and modify the launch criteria (StartInterval or StartCalendarInterval) to your requirements.

Open Terminal, change directory to a location where you want to create the project directory and create a project directory:

> mkdir DesktopprManagePkg
> cd DesktopprManagePkg

Then create a payload and a scripts directory in this project folder:

> mkdir payload
> mkdir scripts

Unzip the desktoppr binary from the downloaded zip archive into the right folder hierarchy in the payload folder:

> mkdir -p payload/usr/local/bin/
> ditto -x -k ~/Downloads/desktoppr-0.5-218.zip payload/usr/local/bin/

Remove the quarantine flag from the expanded desktoppr binary and test it by getting the version and the current wallpaper/desktop picture:

> xattr -d com.apple.quarantine payload/usr/local/bin/desktoppr
> payload/usr/local/bin/desktoppr version
0.5
> payload/usr/local/bin/desktoppr
/System/Library/CoreServices/DefaultDesktop.heic

Create the /Library/LaunchAgents directory in the payload and copy or move the LaunchAgent configuration plist there:

> mkdir -p payload/Library/LaunchAgents/
> cp ~/Downloads/com.scriptingosx.desktopprmanage.plist payload/Library/LaunchAgents/

When the installation package runs, the files in the payload will be moved to the respective locations on the target drive. The pkgbuild tool will set the owner of the files to root when building the package, but we should verify that the file mode is correct:

> stat -f %Sp payload/usr/local/bin/desktoppr
-rwxr-xr-x
> stat -f %Sp payload/Library/LaunchAgents/com.scriptingosx.desktopprmanage.plist
-rw-r--r--

If the privileges don’t match the desired mode, set them with chmod

> chmod 755 payload/usr/local/bin/desktoppr
> chmod 644 payload/Library/LaunchAgents/com.scriptingosx.desktopprmanage.plist

and run the stat commands from above again to verify.

If you built the installer package now, it will install the files in the right locations and the LaunchAgent will load and run desktoppr manage on next login. This will set the desktop/wallpaper according to the information in the configuration profile. You should install and update the configuration profile using your management system.

Loading the LaunchAgent or LaunchDaemon

However, as I said earlier, requiring or waiting for a logout or reboot is not ideal, we would like to have the LaunchAgent load immediately when the package is installed. We can achieve this by adding a postinstall script to the installation package. This script will be executed after the payload files have been installed.

With your favored text editor, create a file named postinstall (no file extension) with this content in the scripts sub directory.

#!/bin/sh

export PATH=/usr/bin:/bin:/usr/sbin:/sbin

label="com.scriptingosx.desktopprmanage"
launchdPlist="/Library/LaunchAgents/${label}.plist"

# do not load when target volume ($3) is not current startup volume
if [ "$3" != "/" ]; then
    echo "not installing on startup volume, exiting"
    exit 0
fi

# for LaunchAgent, check if user is logged in
# see https://scriptingosx.com/2019/09/get-current-user-in-shell-scripts-on-macos/

currentUser=$(echo "show State:/Users/ConsoleUser" | scutil | awk '/Name :/ { print $3 }')

# don't load at loginwindow 
if [ "$currentUser" = "loginwindow" ] || [ "$currentUser" = "_mbsetupuser" ]; then
    echo "no user logged in, exiting"
    exit 0
fi

uid=$(id -u "$currentUser")

echo "loading $label"
launchctl bootstrap "gui/$uid" "$launchdPlist"

Set the proper file privileges on the postinstall file with

> chmod 755 scripts/postinstall

First the script sets the PATH. Then some variables for the label and bath to the label. You can change these to adapt the script to other LaunchAgents or LaunchDaemons.

Then the scripts checks the $3 argument. The installer system passes the target volume to the postinstall script in this argument. While it is a not common in deployment workflows any more, you can still install a pkg on a target volume that is not the current boot volume “/“. We remain safe with our script and check for that possibility, and exit without loading the agent.

LaunchAgents have to be loaded or bootstrapped into the current user’s context. The script gets the current user and checks whether the system might be sitting at the login window. We exit without loading if the system is at the login window or if the current user is _mbsetupuser which means the package is being installed while the system is sitting at Setup Assistant during first setup. Now that the LaunchAgent plist is in place, it will be loaded the first time the user logs in.

When there is a user logged in, we bootstrap the LaunchAgent right away. This is where the modern launchctl syntax shows its advantages. With the legacy commands, this would be:

launchctl asuser "$uid" launchctl load "$launchdPlist"

When you load a LaunchDaemon in a postinstall, the script is much simpler, as you don’t need to check for a current user. You can directly load the daemon into the system target domain:

launchctl bootstrap system/ "$launchdPlist"

Unloading before installation

When you are updating existing software with a package, you have to consider the situation where a LaunchAgent or LaunchDaemon is already running. In this case, we should unload before the installation. We can add a preinstall script (also no file extension) to the scripts folder that will be run before the payload is installed. This closely parallels the postinstall script.

#!/bin/sh

export PATH=/usr/bin:/bin:/usr/sbin:/sbin

label="com.scriptingosx.desktopprmanage"
launchdPlist="/Library/LaunchAgents/${label}.plist"

# do not unload when target volume ($3) is not current startup volume
if [ "$3" != "/" ]; then
    echo "not installing on startup volume, exiting"
    exit 0
fi

if ! launchctl list | grep -q "$label"; then
    echo "$label not loaded, exiting"
    exit 0
fi

# for LaunchAgent, check if user is logged in
# see https://scriptingosx.com/2019/09/get-current-user-in-shell-scripts-on-macos/

currentUser=$(echo "show State:/Users/ConsoleUser" | scutil | awk '/Name :/ { print $3 }')

# don't unload at loginwindow or Setup Assistant
if [ "$currentUser" = "loginwindow" ] || [ "$currentUser" = "_mbsetupuser" ]; then
    echo "no user logged in, exiting"
    exit 0
fi

uid=$(id -u "$currentUser")

echo "loading $label"
launchctl bootout "gui/$uid" "$launchdPlist"

Again, set the proper file privileges on the preinstall file with

> chmod 755 scripts/preinstall

There is one extra step compared to the postinstall script. The preinstall uses launchctl list to check if the launchd configuration is loaded, before attempting to unload it.

Building the Package Installer

Now we get to put everything together. You can use pkgbuild to build an installer package:

> pkgbuild --root payload/ --scripts scripts/ --version 1 --identifier com.scriptingosx.desktopprmanage --install-location / DesktopprManage-1.pkg   
pkgbuild: Inferring bundle components from contents of payload/
pkgbuild: Adding top-level preinstall script
pkgbuild: Adding top-level postinstall script
pkgbuild: Wrote package to DesktopprManage-1.pkg

This works well, but it is difficult to memorize all the proper arguments for the pkgbuild command. If you forget or misconfigure one of them it will either fail outright or lead to a pkg installer file with unwanted behavior. I prefer to put the command in a script. Create a script file buildDesktopprManages.sh at the top of your project directory with these contents:

#!/bin/sh

export PATH=/usr/bin:/bin:/usr/sbin:/sbin

pkgname="DesktopprManage"
version="1.1"
identifier="com.scriptingosx.${pkgname}"
install_location="/"
minOSVersion="10.13"

# determine enclosing folder
projectfolder=$(dirname "$0")

# build the component package
pkgbuild --root "${projectfolder}/payload" \
         --identifier "${identifier}" \
         --version "${version}" \
         --ownership recommended \
         --install-location "${install_location}" \
         --scripts "${projectfolder}/scripts" \
         --min-os-version "${minOSVersion}" \
         --compression latest \
         "${pkgname}-${version}.pkg"

This way, you do not have to memorize the pkgbuild command and its arguments and it is easier to modify parameters such as the version. Remember to change the version as you build newer versions of this package going forward!

Some management systems deploy a pkg file using certain MDM commands. Then you need a signed distribution package instead of a simple component pkg file. (You can learn more about the different types of packages in this blog post or this MacDevOps YVR presentation. Building a sign distribution pkg is somewhat more involved. I have a template script for that in the repo.

Conclusion

As I said, building a LaunchAgent for desktoppr manage is not trivial. However, building LaunchAgents and LaunchDaemons and proper installer pkgs to deploy them is an important skill for Mac admins.

There are some other tools available, such as outset, or your management system might have options to run scripts at certain triggers. But in general, I recommend to understand the underlying processes and technologies before using tools, even when the tools are great and a good fit for the task at hand.

You can use this process of building a LaunchAgent and installer pkg for desktoppr as a template for your own scripts and projects. You can find all the files in the desktoppr repo example directory.

zsh scripts and Root Escalations

There was an update for the Support.app this week which fixed a CVE. There is an argument to be had about whether this CVE deserves its high rating, but it is worth discussing the underlying issue and presenting some solutions. (Erik Gomez has some great comments on Mac Admins Slack.)

zsh is far more configurable than most other shells, most certainly more configurable than the aging bash 3.2 that comes with macOS. One aspect of being more configurable is that it has multiple different configuration files that are loaded at different times when the shell starts. The most common configuration file you will have encountered is ~/.zshrc which is loaded for all interactive shells.

Another, less commonly used, configuration file, ~/.zshenv (and its global sibling /etc/zshenv) is loaded every time a zsh process launches, including script launches. That means, when you (or something else in the system) launches a script with a #!/bin/zsh shebang, the zsh process will read the files /etc/zshenv and ~/.zshenv when they exist, and execute their code.

Update 2024-03-22: some links for further reading:

The Setup

We can see this in action.

First, create a simple zsh “hello.sh” type script with this content:

#!/bin/zsh
echo "Hello, $(whoami)"

Make the script file executable (chmod +x hello.sh) and run it. You should see output like

% ./hello.sh
Hello, armin

where armin is replaced with your current username.

Next, create a new file ~/.zshenv (~/ means at the root of your home directory) with your favored text editor and add the following line:

echo "zshenv: as $(whoami) called from $0"

If you already have a ~/.zshenv you will want to rename it for now so we don’t modify that. (mv ~/.zshenv ~/.zshenv_old)

Note that the configuration file neither has a shebang, nor does it need to be executable.

When you open a new Terminal window, you should see the line

zshenv: as armin called from -zsh

among the other output at the top of the Terminal window, since ~/.zshenv is read and evaluated every time a zsh process starts. The shebang at the beginning of the script ensures a new zsh environment is created for it, even when your interactive shell is not zsh.

When you run hello.sh you will see that ~/.zshenv is executed as well:

% ./hello.sh
zshenv: as armin called from /bin/zsh
Hello, armin

So far, so good. This is how zshenv is supposed to work. It’s purpose is to contain environment variables and other settings that apply to all processes on the system. On macOS this is undermined by the fact that most apps and process are not started from a shell, so they don’t see environment variables set anywhere in the zsh (or another shell’s) configuration files, so .zshenv is rarely used. .zshrc is far more useful.

But now we come to escalation aspect: run hello.sh with root privileges using sudo:

% sudo ./hello.sh
zshenv: as root called from /bin/zsh
Hello, root

We see that both our script and ~/.zshenv are run with root privileges. Again, that is how sudo and .zshenv are supposed to work.

Since the zshenv configuration files are evaluated every time a zsh is launched, it is far more likely that they will eventually run with elevated privileges, than the other configuration files. Other shells, like bash and sh have no equivalent configuration file that gets run on every launch.

The Escalation

The potential danger is the following: a process only needs user privileges to create or modify ~/.zshenv. A malicious attacker can inject code into your .zshenv, wait patiently for a script with a zsh shebang to be run with root privileges, and then execute some malicious code with root privileges on your system.

This can also occur with preinstall or postinstall scripts with a zsh shebang in installation package files (pkg files) when the installation is initiated by the user. A malicious attacker could scan your system for apps where they know the installer package contains zsh scripts, which makes the chance that eventually a zsh script is run with root permissions quite certain. Then all they have to do is be patient and wait…

As root escalations go, this is not a very efficient one. The attacker is at the mercy of the user to wait when they execute a zsh script with escalated root privileges. On some Macs, that may very well be never.

This also only works when the user can gain administrative privileges. Standard users cannot use sudo to gain root privileges, or run installation packages.

There are more effective means of escalating from user to root privileges on macOS. Most easily by asking the user directly for the password with a dialog that appears benign. However, if you are a frequent user or author of zsh scripts, it is important to be aware.

From what I can tell so far, this does not affect scripts launched from non-user contexts, such as scripts from Munki, Jamf Pro or other management solutions, though there may be odd, unexpected edge cases.

As an organization, you can monitor changes to all shell configuration files with a security tool (such as Jamf Protect). There are more shenanigans an attacker could achieve by modifying these files. There are, however, many legit reasons to change these files, so most changes will not indicate an attack or intrusion. Nevertheless, the information could be an important puzzle piece in combination with other behaviors, when putting together the progress or pattern of an intrusion.

Update 2024-03-22: Matteo Bolognini has added a custom analytic for Jamf Protect to detect changes to the zshenv file.

Configuration file protection

This weakness is fairly straightforward to protect against. First, since the attack requires modification of ~/.zshenv, you can protect that file.

If you did not have a ~/.zshenv, create an empty file in its place or remove the line of code from our sample .zshenv before. Then apply the following flag:

% sudo chflags schg ~/.zshenv

This command sets the system immutable flag (schg, ‘system change’), which makes it require root privileges to modify, rename or delete the file.

If you want to later modify this file, you can unlock it with

% sudo chflags noschg ~/.zshenv

Just remember to protect it again when you are done.

Script protection

Since most Mac users will never open terminal and never touch any shell configuration file, relying on protecting the configuration file is not sufficient. However, it is quite easy to protect the zsh scripts you create from this kind of abuse.

One solution is to use a $!/bin/sh shebang instead of #!/bin/zsh. Posix sh does not have a configuration file which gets loaded on every launch, so this problem does not exist for sh.

If you are using zsh features that are not part of the POSIX sh standard you will need to change the script. In my opinion, installation scripts that are too complex to use POSIX sh, are attempting to do too much and should be simplified, anyway. So, this can also work as an indicator for “good” installation scripts.

In addition, installer packages might be used in situations where zsh is not available. Installation tools that install package files when the Mac is booted to Recovery (where there is no zsh) are used far less than they used to, but it is still possible and making your installer pkgs resilient to all use cases is generally a good choice.

I believe for installation package scripts, using an sh shebang is a good recommendation.

Some scripts popular with Mac Admins, like Installomator, do not usually run from installer packages, but are regularly run with root privileges. For these cases, (or for installation scripts, where cannot or do not want to use sh) you can protect from the above escalation, by adding the --no-rcs option to the shebang:

#!/bin/zsh --no-rcs
echo "Hello, $(whoami)"

zsh’s --no-rcs option suppresses the launch of user level configuration files. The rc stands for ‘Run Command’ files and is the same rc that appears in zshrc or bashrc or several other unix configuration files.

That allows you to keep using zsh and zsh features, but still have safe scripts. This is the solution that Support.app and the latest version of Nudge have implemented. I have also created a PR for Installomator, which should be merged in the next release.

Conclusion

Modifying ~/.zshenv is not a very effective means of gaining root privileges, but it is something developers and Mac admins that create zsh script that may be run with root privileges should be aware of. Switch to an sh shebang or add --no-rcs to the zsh shebang of scripts that might be run with root privileges to protect.

Installomator v10.3 and v11.0beta1

We have released a new minor version of Installomator. Version 10.3 contains several new labels and some very important fixes to existing labels. You can see the details in the v10.3 release notes.

We have also released a first beta for the v11.0 release.

Having a beta parallel to a new minor release is a new approach for us. The way Installomator is built, changes and additions to labels do not (well, should not) affect the hundreds of other labels. This allows us to add and update labels quite easily. (We have added and updated 205 labels since v9.2, and 37 labels since 10.2) The minor updates focus on adding new labels and updating the existing labels, mostly because the names or download URLs change on the vendor side. Since not everyone is comfortable with the intricacies of git and GitHub, frequent minor releases are important to keep Installomator working for everyone that uses it.

However, we want to update the script and functionality in the script, as well. But since any change to behavior of the main script might affect all 500+ applications, we have to tread very carefully here. Last year, we had an extended beta period for v10, which was necessary to identify some problems with the changes. However, we didn’t release new minor updates during that beta phase which means that many labels in the v9.2 release broke over the beta phase.

Because of this, I have studied some new git and GitHub skills. Now, there will be a minor release with new and updated labels, as well as a new beta for v11.0 with some new features, that we are quite excited about. We will keep this up until we deem v11 to be ready for production.

Do not use v11.0beta1 in production! That is what the v10.3 release is for. But please, test the beta in your testing environments and report all issues that you find. This will help us build a better, safe, and stable Installomator v11.0.

As always, many thanks to everyone who is helping to make this project so much better than I could have ever imagined…

Update: Installomator v10.1

Minor update to Installomator, which brings it to version 10.1. Added and updated a bunch of labels. Many thanks to all who contributed!

  • updated Jamf/Dialog scripts icon handling (#778)
  • Readme Updates (#744)
  • new labels:
    • amazoncorretto11jdk (#721)
    • amazoncorretto17jdk (#721)
    • bbeditpkg (#720)
    • boop (#781)
    • camtasia2021, camtasia2022 (#730)
    • jamfcpr (#753)
    • jetbrainsrider
    • lgcalibrationstudio (#763)
    • mendeleyreferencemanager (#713)
    • microsoftofficefactoryreset (#751)
    • microsoftofficeremoval (#755)
    • mist-cli (#733)
    • mist (#732)
    • mobiletolocal (#752)
    • netiquette (#770)
    • todoist (#769)
    • transfer (#773)
    • vpntracker365 (#760)
    • zerotier (#785)
  • updated labels:
    • 1password8 (#759)
    • amazoncorretto8jdk (#721)
    • camtasia (#730)
    • citrixworkspace (#731)
    • code42 (#766)
    • drawio (#725)
    • duodevicehealth (#761)
    • idrive (#726)
    • idrivethin (#727)
    • macfuse (#714)
    • microsoftazuredatastudio (#788)
    • nudge (#754)
    • prism9 (#746)
    • skype (#762)
    • synologydriveclient (#789)
    • ultimakercura (#740)

MDOYVR 22 Talk: The Encyclopedia of macOS Automation

Last week I had the pleasure and honor of participating and presenting at MacDevOps YVR. The videos for the sessions are now appearing on YouTube.

There is a page for my talk “The Encyclopedia of macOS Automation,” in which I discuss the options for scripting and automation on macOS, with extra links and notes. You can go directly to the video here.

The talks this year were graphic recorded by the amazing Ashton Rodenhiser (website, twitter). The graphic at the top of this post was made by her while I was presenting.

As always, I had a lot of fun at this conference. Many thanks to the organizers and all the other speakers. Until next year!

Update Installomator: v9.2

We have updated Installomator. This brings Installomator to 465(!) applications! Many thanks to everyone who contributed.

Note: Both Google and Mozilla recommend using the pkg installers instead of the dmg downloads for managed deployments. So far, Installomator has provided labels for both. (googlechrome and googlechromepkgor firefox and firefoxpkg, respectively) Since there are problems with the dmg downloads, a future release of Installomator will disable the firefox and googlechrome dmg labels. You should switch to using the firefoxpkg or googlechromepkg labels instead.

  • bug and documentation fixes
  • 40 new, and 26 updated labels

You can find more details in the release notes.

Launching Scripts #4: AppleScript from Shell Script

In the last post, we discussed how to run shell commands and scripts from an Apple Script environment. In this post, we will look at how we can run AppleScript commands and scripts from the shell environment.

Open Scripting Architecture

The key to running AppleScript from the shell is the osascript command. OSA is short for ‘Open Scripting Architecture’ which is the framework that powers AppleScript. This framework allows AppleScript to have its native language, but also use JavaScript syntax.

The osascript command allows us to run AppleScript commands from Terminal and shell. The most common use is the user interaction commands from AppleScript, like display dialog:

osascript -e 'display dialog "Hello from shell"'

The -e option tells osascript that it will get one or more lines of statements as arguments. The following argument is AppleScript code. You can have multiple -e options which will work like multiple lines of a single AppleScript:

> osascript -e 'display dialog "Hello from shell"' -e 'button returned of result'
OK

osascript prints the value of the last command to stdout. In this case, it is the label of the button clicked in the dialog. (The ‘Cancel’ button actually causes the AppleScript to abort with an error, so no label will be returned for that.)

When you have multiple lines of script, using multiple -e statements will quickly become cumbersome and unreadable. It is easier to use a heredoc instead:

osascript <<EndOfScript
   display dialog "Hello from shell"
   return button returned of result
EndOfScript

This also avoids the problem of nested quotation marks and simplifies shell variable substitution.

Shell variables and osascript

There are a few ways to pass data into osascript from the shell.

Since the shell substitutes variables with their value before the command itself is actually executed, this works in a very straightforward manner:

computerName=$(scutil --get ComputerName)

newName=$(osascript -e "text returned of (display dialog \"Enter Computer Name\" default answer \"$computerName\")")

echo "New Name: $newName"

This works well, but because we want to use shell variable substitution for the $computerName, we have to use double quotes for the statement. That means we have to escape the internal AppleScript double quotes and everything starts to look really messy. Using a heredoc, cleans the syntax up:

computerName=$(scutil --get ComputerName)

newName=$(osascript <<EndOfScript
    display dialog "Enter Computer Name" default answer "$computerName"
    return text returned of result
EndOfScript
)

echo "New name: $newName"

I have a detailed post: Advanced Quoting in Shell Scripts.

Environment Variables

Generally, variable substitution works well, but there are some special characters where it might choke. A user can put double quotes in the computer name. In that case, the above code will choke on the substituted string, since AppleScript believes the double quotes in the name end the string.

If you have to expect to deal with text like this, you can pass data into osascript using environment variables, and using the AppleScript system attribute to retrieve it:

computerName=$(scutil --get ComputerName)

newName=$(COMPUTERNAME="$computerName" osascript <<EndOfScript
    set computerName to system attribute "COMPUTERNAME"
    display dialog "Enter Computer Name" default answer computerName
    return text returned of result
EndOfScript
)

echo "New name: $newName"

The shell syntax

VAR="value" command arg1 arg2...

sets the environment variable VAR for the process command and that command only. It is very useful.

Retrieving environment variables in AppleScript using system attribute is generally a good tool to know.

Interpret this!

osascript can also work as a shebang. That means you can write entire scripts in AppleScript and receive arguments from the shell. For example, this script prints the path to the front most Finder window:

#!/usr/bin/osascript

tell application "Finder"
    if (count of windows) is 0 then
        set dir to (desktop as alias)
    else
        set dir to ((target of Finder window 1) as alias)
    end if
    return POSIX path of dir
end tell

You can save this as a text file and set the executable bit. I usually use the .applescript extension.

> print_finder_path.applescript
/Users/armin/Documents

To access arguments passed into a script this way, you need to wrap the main code into a run handler:

#!/usr/bin/osascript

on run arguments
    if (count of arguments) is 0 then
        error 2
    end if
    return "Hello, " & (item 1 of arguments)
end

You can combine this into a longer script:

macOS Privacy and osascript

When you ran the above script, you may have gotten this dialog:

If you didn’t get this dialog, you must have gotten it at an earlier time and already approved the access.

AppleEvents between applications are controlled by the macOS Privacy architecture. Without this, any process could use AppleEvents to gather all kinds of data from any process. These dialogs are easy enough to deal with when running from Terminal. But if you put your AppleScript code (or shell scripts calling AppleScript) into other apps or solutions, it could get messy quite quickly.

Mac Admins generally want their automations to run without any user interactions. You can avoid these dialogs by creating PPPC (Privacy Preferences Policy Control) profiles that are distributed from an MDM server. In this case you have to pre-approve the application that launches the script, which can sometimes also be challenge. The other option is to find solutions that avoid sending AppleEvents altogether.

I have a longer post detailing this: Avoiding AppleScript Security and Privacy Requests

osascript and root

Management scripts often run as a privileged user or root. In this case, certain features of AppleScript may behave strangely, or not at all. I generally recommend to run osascript in the user context, as detailed in this post: Running a Command as another User

Conclusion

AppleScript’s bad reputation may be deserved, because its syntax is strange, and often very inconsistent. Nevertheless, it has features which are hard to match with other scripting languages. You can use the strategies from this and the previous posts to combine AppleScript with Shell Scripting and other languages to get the best of both worlds.

Launching Scripts #3: Shell scripts from AppleScript

In this series of posts, I am exploring the various different ways of launching scripts on macOS. In the first two posts, we explored what happens when you launch scripts from Terminal. We already explored some concepts such as the shell environment and how that affects scripts. In this post we are going to explore a different, but very common way to launch shell commands and scripts: AppleScript’s do shell script command.

do shell script

When AppleScript made the transition from Classic Mac OS 9 and earlier to Mac OS X, it gained one command that allowed AppleScripts to interact with Mac OS X’s Unix environment. The do shell script command executes a command or script in the shell and returns the output as a text to the AppleScript.

We have used this in an earlier post:

do shell script "echo $PATH"
    --> "/usr/bin:/bin:/usr/sbin:/sbin"

But you can use this to run any shell command:

do shell script "mdfind 'kMDItemCFBundleIdentifier == org.mozilla.firefox'"
    --> "/Applications/Firefox.app"

Note the use of single quotes inside the double quotes which delineate the AppleScript text. I have a post on the challenges of quoting in these mixed scripting environments.

You can assemble the command you pass into do shell script using AppleScript text operators:

set bundleID to "org.mozilla.firefox"
do shell script "mdfind 'kMDItemCFBundleIdentifier == " & bundleID & "'"
    --> "/Applications/Firefox.app"

Note that the PATH variable for AppleScripts that are run from Script Editor or as an AppleScript applet is different than the PATH in your interactive environment. Most notably, it does not include /usr/local/bin. When you want to use a command or script that is not stored in the four default directories, you will have to use the full path in the do shell script:

do shell script "/usr/local/bin/desktoppr"

(Desktoppr is a small tool I built to work with desktop pictures on macOS, you can get it here.)

When you are unsure what the full path to a command is, you can use the which command in Terminal:

> which desktoppr
/usr/local/bin/desktoppr

Errors

Keep in mind that which uses the same logic to lookup a command as the shell does when it looks up a command with no path. So, if you think you can trick AppleScript by using the which command to lookup a non-standard command, it will still fail:

do shell script "which desktoppr"
    --> error "The command exited with a non-zero status." number 1

When the command in do shell script returns a non-zero exit code, you will get an interactive dialog informing the user of the error. The AppleScript will not continue after the error. You can handle the error the same way you would handle any AppleScript error, with a try… on error block:

set filepath to "/unknown/file"

try
    do shell script "/usr/local/bin/desktoppr" & quoted form of filepath
on error
    display alert "Cannot set desktop picture to '" & filepath & "'"
end try

Files and Paths

AppleScript has its own methods of addressing files and folders. Actually, there are multiple ways, which is one of the confusing things about AppleScript. Neither of the native forms of addressing files and folder in AppleScript use the standard Unix notation with forward slashes separating folders. But there are built-in tools to convert from Unix notation to AppleScript and back.

Use the POSIX path attribute to get a Unix style file path from an AppleScript file or alias. Unix style paths used with commands need spaces and other special characters escaped. You can use the quoted form attribute to escape any AppleScript string. This is usually used directly with POSIX path:

set imagefile to choose file "Select a Desktop"
    --> alias "Macintosh HD:Library:Desktop Pictures:BoringBlueDesktop.png"
set imagepath to quoted form of POSIX path of imagefile
    --> '/Library/Desktop Pictures/BoringBlueDesktop.png'
do shell script "/usr/local/bin/desktoppr " & imagepath

You can convert a Unix style file path into an AppleScript file with the POSIX file type:

set bundleID to "org.mozilla.firefox"
set appPaths to do shell script "mdfind 'kMDItemCFBundleIdentifier == " & bundleID & "'"
    --> "/Applications/Firefox.app"

if (count of paragraphs of appPaths) = 0 then
    display alert "No app found"
else
    set appPath to first paragraph of appPaths
    -- convert the path to an AppleScript file
    set appFile to POSIX file appPath
        --> file "Macintosh HD:Applications:Firefox.app:"
    tell application "Finder" to reveal appFile
end if

Shell Scripts in AppleScript Bundles

Sometimes you are writing an AppleScript and want to use a script for some functionality which is difficult to achieve in AppleScript. When you have an AppleScript file and a shell script file that work together this way, you want to store them together and also have an easy way for one to get the location of the other.

For this, AppleScript provides the notion of script bundles and AppleScript applets (which are also bundles). A script bundle is not a flat file, but a folder which contains the script itself and other resources, such as script libraries, or shell scripts. The script can easily locate items in its bundle and call them.

For example, we want a script that needs to unzip a file. We can use the unzip command line tool to do that, but in my experience it is better to use ditto. The ditto expansion seems to be closer to how the expansion from Archive Utility works and do better with extended attributes and resource forks and other macOS specific things.

This is the shell script for our example:

#!/bin/sh

# target dir for expansion
targetdir="/Users/Shared/Script Bundle Demo"

# sanity checks for argument 1/filepath
if [ -z "$1" ]; then
    exit 2
fi

filepath=$1

# is it a file?
if [ ! -f "$filepath" ]; then
    exit 3
fi

# note: ditto seems to work better than unzip
ditto -x -k "$filepath" "$targetdir"

This is simple enough that you could just do it in a one-line do shell script, but I want to keep it simple. You could extend this shell script to use different tools to expand different types of archives, such as xar, tar, aa etc. If you want a more complex script, feel free to build one!

Now we can build the AppleScript/shell script combination. Open Script Editor, create a new script and save it right away. In the save dialog, change the ‘File Format’ to ‘Script bundle’ before saving.

After you have saved the Script as a bundle, you can see the Bundle Info in a pane on the right side of the script window. If you don’t see this pane, choose ‘Show Bundle Contents’ from the ‘View’ menu, or click the right most icon in the tool bar.

In this pane, you can set the name, identifier, version and some other data for the script bundle. You can also see a list of the ‘Resources’ which shows the contents of the Contents/Resources folder in the script’s bundle. When you find the script you save, you will see it has a scptd file extension and when you open the context menu on it in Finder, you can choose ‘Show Package Contents’ and dig into the bundle contents.

Note: AppleScript applications (or applets) work the same way. Their .app bundles have a few more sub folders, but the Resources work the same way. The difference is that AppleScript applets work on double-click, drag’n drop, and some other events that we will get to in later posts. Script bundles have to run from Script Editor.

Save the shell script from above into the script bundle’s Resources sub-directory with the name unarchive.sh. You should see it appear in the ‘Resources’ list in the script window.

This way, the AppleScript bundle can contain all the resources it might need, including shell (or other) scripts.

Now we still need to find a way to access the Resources from the script. To run our shell script, add the following code to the AppleScript in Script Editor:

-- Script Bundle Demo
set theArchive to choose file "Select a zip archive:" of type {"zip"}
set archivePath to quoted form of POSIX path of theArchive

-- assemble command
set scriptPath to quoted form of POSIX path of my (path to resource "unarchive.sh")
set commandString to scriptPath & space & archivePath

-- for debugging 
log (commandString)

do shell script commandString

First we prompt the user to choose a file with a zip extension, and the we convert the result into a quoted Unix path.

Then, we use the path to resource "unarchive.sh" to get the path to our shell script in the Resources folder in the bundle. Then we get the quoted Unix notation, and assemble the command for the do shell script. The log command there will print the commandString to the output in the script window and is useful for debugging. Then we run the command with do shell script.

Environment for do shell script

Our example script expands the archive into a subfolder of /Users/Shared. If you wanted to use a different location, you could use a second argument in the script.

There is a different way of passing data into scripts and that is environment variables.

First of all it is important to note that the shell environment for commands and scripts run with the do shell script command from an AppleScript in Script Editor or an AppleScript application is very different from the shell environment in an interactive shell in the Terminal. We have already seen that the PATH environment variable has a different value, which influences the command lookup behavior.

You can check the environment variable by running the env command. This will list all environment variables. (To be nitpicky, there is more to a shell environment than just the env variables, there are also shell options, but those will be different for each shell, sh, bash or zsh, anyway.)

do shell script "env"
    --> "SHELL=/bin/zsh
TMPDIR=/var/folders/2n/q0rgfx315273pb4ycsystwg80000gn/T/
USER=armin
COMMAND_MODE=unix2003
__CF_USER_TEXT_ENCODING=0x1F5:0x0:0x0
PATH=/usr/bin:/bin:/usr/sbin:/sbin
__CFBundleIdentifier=com.apple.ScriptEditor2
PWD=/
XPC_FLAGS=0x0
SHLVL=1
HOME=/Users/armin
LOGNAME=armin
_=/usr/bin/env"

Interestingly, we have USER and HOME to use in this environment.

We can also add environment variables to a do shell script command:

do shell script "TARGET_DIR='/Users/Shared/Script Bundle Demo' " & scriptPath & space & filePath

You can use this to set the value of the TARGET_DIR env variable for the next command, which is our script in the script bundle.

Administrator Privileges

No matter which way you use do shell script, it has one big benefit. You can prompt the user to get the command or script to run with administrative privileges:

do shell script "systemsetup -getRemoteLogin" with administrator privileges

This will prompt for the user name and password of an administrator user. This can allow to build some simple workflows that require user interaction and administrative privileges really easily.

Conclusion

Combining Script Bundles and AppleScript Applications with shell scripts can create a powerful combination. You can use the “best of both worlds” with some of AppleScript’s user interaction commands and the shell’s strength in file manipulation and similar workflows. You can also sign AppleScript applications with a valid Apple Developer ID and pre-approve them for privacy exemptions with a PPPC profile.

If this explanation is not detailed enough for you, there is an amazing Tech Note in Apple’s Documentation Archive.

This post covered launching shell scripts from AppleScript. In the next post we will launch AppleScript code from shell scripts.

Launching Scripts #2: Launching Scripts from Finder

In this series of posts, I will explore the many ways that you can launch a script on macOS. In the previous, inaugural post, I described what happens when you launch a script from an interactive terminal shell.

There are several virtual terminal applications available for macOS. iTerm is very popular. Some text editors like Visual Studio Code and Nova, have terminals built-in. Since the actual launching of an executable is done by the shell running inside the virtual terminal, the launch process remains the same.

That said, Terminal app has a useful trick up its sleeves.

command file extension

When you change the file extension of a script to .command, double-clicking the file will open it in a new Terminal window and run it there. Any input or output the script requires will happen in that Terminal window. When the script exits, the shell session in the Terminal window will exit.

Let’s take this simple script:

#!/bin/sh
echo "Enter your name: "
read -r username
echo "Hello, $username"

When you put this this in a .command file and double-click it, you get a new Terminal window with:

/Users/armin/Desktop/hello_name.command ; exit;                                 
~ % /Users/armin/Desktop/hello_name.command ; exit;
Enter your name: 

You can see that Terminal opens a new window with a new, default shell and all your configurations, then launches the script right away. The script prints its output and then waits for the user input (the read command). When you enter the name at the prompt, the script continues.

Armin
Hello, Armin

Saving session...completed.

[Process completed]

When the script ends, the shell in the Terminal window exits, as well. No more interactive prompt will be shown.

This script expects user input in the Terminal and then presents output to stdout in the same window. While you could re-write a script to use AppleScript’s display dialog to handle both in the input and the output, it would make the script significantly more complex.

Instead, you can change the file extension to .command and then a double-click will create a new Terminal window where the user interaction (input and output) takes place. For the right kind of user and workflow, this can be a sufficient solution with practically no overhead.

You can also remove the file extension completely. The behavior when you double-click such a file in Finder will be the same. Extension-less executables also get a different the icon. Either way, you need to have the executable bit set for the script.

Note: iTerm can also open .command files, but I have had some trouble with user interaction in these cases. Since I usually don’t use iTerm, maybe I have something setup wrong?

Quarantine and Gatekeeper

A Terminal window is not a user interface that many users will appreciate, but this allows you share scripts with other users in a form they understand. “Double-click this to run” is something that fits with most users’ idea of how macOS works.

Before you start creating dozens of .command scripts and share them, there is a major tripwire that macOS security has set up.

When you share an executable file through a website, email or a chat message, macOS will attach a quarantine flag. With applications, this flag triggers a GateKeeper scan before the app is launched and it will show the standard dialog, even when the app is signed and notarized and a much more “scary” warning when it is not.

When you launch a command or script from Terminal, either directly or indirectly with a double-click, and it still has the quarantine flag set, it will not launch. You will not get one of the standard Gatekeeper dialogs. Just an opaque operation not permitted error in the Terminal output.

You can check if a file has the quarantine flag set with the xattr command:

> xattr hello_name.command
com.apple.TextEncoding
com.apple.lastuseddate#PS
com.apple.quarantine 

Your list list of extended attributes may be different. The quarantine flag has the label com.apple.quarantine.

The xattr command also can remove the quarantine flag:

> xattr -d com.apple.quarantine hello_name.command

Apple seems to assume that when you are using Terminal, you know what you are doing. That means you can bypass most of the mechanisms that attach a quarantine flag with command line tools. When you download something with curl it won’t get quarantined. You can install an unsigned, unnotarized pkg installer using the installer command. Because this is possible, it doesn’t mean it is always wise. Piping a curl command directly into sh or bash or any interpreter is still poor security.

Most of the time though, a script file shared to another Mac or another user will almost certainly get the quarantine flag. Users who are comfortable with using Terminal should be able to use the xattr command to disable this protection, but this is not something for ‘normal’ users. So quarantine, makes the use of the command file extensions far less effective than it could be. This is probably intentional, since executables that are opened by double-click can be an easy way to sneak malware and other unwanted software onto a system, leveraging a user’s ignorance of what is happening.

This is generally true when you move scripts and executable files between macOS systems. I have also seen the quarantine flag getting set when you store a script in a cloud sync service (especially iCloud) or when you edit an executable with a sandboxed application.

One way around the quarantine, would be to distribute and properly install the scripts with an installer pkg. Then you can properly sign and notarize the installer pkg and the scripts will not be quarantined, as they come from a trusted and verified source. This may be a good solution for some workflows, but generally feels a bit “over-designed.”

Output flashes by so quickly

There is a setting in Terminal’s preferences which determines what happens with windows when the shell exits. You can find it under the ‘Shell’ tab in the ‘Profiles’ area. This setting can be different for each profile. Under ‘When the shell exits’ there is a popup menu with the options ‘Close the window,’ ‘Close if the shell exited cleanly,’ and ‘Don’t close the window.’ The last ‘Don’t close’ is the default.

When you have this option set to ‘Close the window,’ the new Terminal window from a command file might only be active and visible for a short time. This may or may not be a good thing, depending on what you want.

Conclusion

Script files with the command file extension can be a simple, straightforward way to make scripts easily ‘launchable’ from Finder. You can also put them in the Dock or in the Login Items. The user experience is, well, a terminal, so not terribly nice, but it can be useful, and does not require any modification of the script.

When you share executable scripts, whether they have the command file extension or not, Gatekeeper quarantine on macOS can prevent the script from running. You should get familiar with the quarantine flag and the xattr command to manipulate it.

In the next post, we launch shell scripts from AppleScripts.]