Scripting OS X

In my last post introducing desktoppr 0.5, I said that a Mac admin could build a customized pkg with a customized LaunchAgent property list, that would run desktoppr manage at login and every few hours in the background to re-set the wallpaper when the configuration profile is updated.

I admit I glossed over the process of building a package that installs and launches a LaunchAgent (or LaunchDaemon). Since that is not a trivial configuration, I will describe the details in this post.

LaunchAgent? LaunchDaemon? LaunchD?

On macOS, launchd is the system-wide process that controls and launches every other process running on the system. When you look at the hierarchical process list in Activity Monitor, you will always see launchd running with the process ID 1 as the single child process of kernel_task and all other processes are a child of launchd.

More practically, users and admins can use launchd to control processes that need to be available at all times or be launched in certain intervals or at certain events.

launchd has two types of processes: LaunchDaemons and LaunchAgents. Since the terms daemons (or demons) and agents do not have clear definitions outside of launchd, a bit of explanation is required.

LaunchDaemons

LaunchDaemons are processes that run with system privileges (“as root”). Processes that belong to your management system and security suites may need to run periodically or on certain events in the background and require root level access to the system to do their jobs. Other processes or services need to listen for incoming network traffic on privileged ports.

Note that background process are often called ‘agents’ regardless of whether they run with root privileges or as a user. The launchd terminology is more precise here.

Apple has done a lot of work over the past few year to provide Endpoint Security or Network Extension frameworks that mitigate the need for third party root daemons, but there are still relevant use cases, especially when you are a managing Macs.

A LaunchDaemon is configured by providing a property list file in /Library/LaunchDaemons. This configuration plist will be automatically loaded at reboot, or can be loaded using the launchctl command.

LaunchAgents

LaunchAgents are processes that run with user privileges. Generally, if something needs to read or write to anything in the user home folder, it should be a LaunchAgent. Over the recent years, Apple has put restrictions on what data process can access in user space, which has made LaunchAgents more of a pain to manage, but they still have their use cases.

A LaunchAgent is configured by providing a property list file in /Library/LaunchAgents on the system level or in ~/Library/LaunchAgents in a user’s home folder. Configuration property lists that are in a user home folder, will only run for that user, while LaunchAgents that are configured in the system wide /Library/LaunchAgents will be loaded for all users on the system.

A LaunchAgent configuration plist will be automatically loaded at login, or can be loaded using the launchctl command.

Since desktoppr affects a user setting (the desktop/wallpaper), it has to run as the user, so we need to build and install a LaunchAgent. Creating and installing a LaunchDaemon is pretty much the same process, though.

Ingredients

To build an installation package to install and run a LaunchAgent or LaunchDaemon, you need three ingredients:

• the binary or script which will be launched
• a launchd configuration property list
• installation scripts to unload and load the configuration

For desktoppr, we already have the binary (downloadable as a zip from the project release page) and a sample configuration property list file.

The sample launchd configuration looks like this:

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
    <key>Label</key>
    <string>com.scriptingosx.desktopprmanage</string>
    <key>ProgramArguments</key>
    <array>
        <string>/usr/local/bin/desktoppr</string>
        <string>manage</string>
    </array>
    <key>RunAtLoad</key>
    <true/>
    <key>StartInterval</key>
    <integer>10800</integer>
</dict>
</plist>

The label is required and is used to uniquely identify the LaunchAgent or Daemon with launchd and the launchctl command. Reverse domain notification is common. The filename of the configuration property list should match the label.

The ProgramArguments key provides an array with the command and arguments that will be launched. The first item in the array should provide the full path to the binary or script. Then you provide arguments, one string item per argument. Since the individual arguments are separated by the string tags, you do not need to escape spaces or other special characters in the arguments.

It is worth noting that launchd will launch the binary or script directly, so syntax and substitutions that are done in shells will not work. This includes variable and command substitution like $HOME, $USER, or $(date) as well as pipes and output redirection. You get a single command with a list of static arguments. (You can use the StandardOutPath and StandardErrorPath keys in the launchd config plist to redirect output.)

For the desktoppr LaunchAgent, that suits us just fine, as we are only passing the manage argument which tells desktoppr to get the details from a configuration profile (sample here).

The next key, RunAtLoad tells launchd, to run the binary with the arguments immediately when the configuration file is loaded. Since configuration files in /Library/LaunchAgents and ~/Library/LaunchAgents are automatically loaded when a user logs in, this generally means desktoppr manage will be run when a user logs in, which suits us well.

The last key StartInterval tells launchd to re-launch the process after a certain time (given in seconds, the 10800 seconds in our sample file translate to three hours). Should the system be sleeping at the time, it will not run the LaunchAgent at that time, or when the system wakes up, but wait until the next interval period comes around.

There are other keys that control when the process gets launched, such as StartCalendarInterval or WatchPath. You can read details in the man page for launchd.plist or on the excellent launchd.info page. There are also apps, like Peter Borg’s Lingon that provide a user interface for creating these plist files.

Move or copy the plist file into the correct folder. The plist files in /Library/LaunchDaemons and /Library/LaunchAgents must be owned by root. They must not be writable by group or other. The file mode 644 (rw-r--r--) is recommended. Similarly, the binary or script needs to be owned by root and cannot be writable by group or other. The file mode 755 (rwxr-xr-x) is recommended. Wrong file privileges will result in a Load/Bootstrap failed: 5: Input/output error.

Make sure that the binary or script does not have a quarantine flag attached. You can check with ls -al@ or xattr -p com.apple.quarantine /path/to/file and remove a quarantine flag with xattr -d com.apple.quarantine /path/to/file. When the binary or script has the quarantine flag, the configuration file will load fine but the actual execution will fail quietly.

Loading the LaunchAgent or LaunchDaemon

The easiest way to load a LaunchDaemon is to restart the Mac. The easiest way to load a LaunchAgent is to log out and login. This is usually not practical.

The launchctl command manages LaunchDaemons and LaunchAgents and we can use this to load LaunchAgents and LaunchDaemons on demand.

To load our desktoppr agent, use

> launchctl load /Library/LaunchAgents/com.scriptingosx.desktopprmanage.plist

People are already readying their pitchforks here. “Hold on,” they say. “Apple has marked load and unload as ‘legacy.’ You shouldn’t use them!”

While it is true that load and unload (and a bunch of other commands) are labeled as ‘legacy’ in the launchctl man page, this does not mean they are deprecated and should be avoided. The difference between the legacy and the ‘new’ commands is that the legacy commands pick up whether a process should be run in user or root context (the ‘target domain’) from the context that launchctl runs in, whereas the new bootstrap and bootout commands need the target domain to be stated explicitly.

This makes the modern commands more precise, but often more wordy to use. You will see that the legacy commands are simpler to use in the interactive terminal, while the ‘modern’ commands are more useful in scripts. The equivalent ‘modern’ command is:

> launchctl bootstrap gui/501 /Library/LaunchAgents/com.scriptingosx.desktopprmanage.plist

where 501 is the user ID for your user account. This is 501 by default for the first user on on a macOS system, but you should verify what your account’s user ID actually is with id -u.

See what I mean with “more wordy to use”? That doesn’t mean they are bad or should be avoided, but that we can be picky about when to use which format. You could argue that using the modern commands throughout would be more consistent, but you still find the ‘legacy’ command is lots of documentation, so I feel a Mac admin needs to be aware of both.

Note: the ‘modern’ commands were introduced in OS X Yosemite 10.10 in 2014.

To load a LaunchDaemon, you need to run launchctl with root privileges. In the interactive shell, that means with sudo:

> sudo launchctl load /Library/LaunchDaemons/com.example.daemon.plist

or

> sudo launchctl bootstrap system/ /Library/LaunchDaemons/com.example.daemon.plist

To stop a LaunchAgent from being launched going forward, unload the configuration:

> launchctl unload /Library/LaunchAgents/com.scriptingosx.desktopprmanage.plist

or

> launchctl bootout gui/501/com.scriptingosx.desktopprmanage

The bootout command uses the label instead of the file path.

Putting it all together in an installer

Note: I will be showing how to build and installer package for our LaunchAgent using command line tools. These instructions should have all the information you need, even if you prefer using apps such as Whitebox Packages or Composer.

If you are not comfortable with using Terminal on macOS yet, please consider my book “macOS Terminal and Shell.” If you want to learn more about building installation packages, consider my book “Packaging for Apple Administrators.”

Download the zip file for desktoppr from the releases page. Download the sample launchd property list file and modify the launch criteria (StartInterval or StartCalendarInterval) to your requirements.

Open Terminal, change directory to a location where you want to create the project directory and create a project directory:

> mkdir DesktopprManagePkg
> cd DesktopprManagePkg

Then create a payload and a scripts directory in this project folder:

> mkdir payload
> mkdir scripts

Unzip the desktoppr binary from the downloaded zip archive into the right folder hierarchy in the payload folder:

> mkdir -p payload/usr/local/bin/
> ditto -x -k ~/Downloads/desktoppr-0.5-218.zip payload/usr/local/bin/

Remove the quarantine flag from the expanded desktoppr binary and test it by getting the version and the current wallpaper/desktop picture:

> xattr -d com.apple.quarantine payload/usr/local/bin/desktoppr
> payload/usr/local/bin/desktoppr version
0.5
> payload/usr/local/bin/desktoppr
/System/Library/CoreServices/DefaultDesktop.heic

Create the /Library/LaunchAgents directory in the payload and copy or move the LaunchAgent configuration plist there:

> mkdir -p payload/Library/LaunchAgents/
> cp ~/Downloads/com.scriptingosx.desktopprmanage.plist payload/Library/LaunchAgents/

When the installation package runs, the files in the payload will be moved to the respective locations on the target drive. The pkgbuild tool will set the owner of the files to root when building the package, but we should verify that the file mode is correct:

> stat -f %Sp payload/usr/local/bin/desktoppr
-rwxr-xr-x
> stat -f %Sp payload/Library/LaunchAgents/com.scriptingosx.desktopprmanage.plist
-rw-r--r--

If the privileges don’t match the desired mode, set them with chmod

> chmod 755 payload/usr/local/bin/desktoppr
> chmod 644 payload/Library/LaunchAgents/com.scriptingosx.desktopprmanage.plist

and run the stat commands from above again to verify.

If you built the installer package now, it will install the files in the right locations and the LaunchAgent will load and run desktoppr manage on next login. This will set the desktop/wallpaper according to the information in the configuration profile. You should install and update the configuration profile using your management system.

Loading the LaunchAgent or LaunchDaemon

However, as I said earlier, requiring or waiting for a logout or reboot is not ideal, we would like to have the LaunchAgent load immediately when the package is installed. We can achieve this by adding a postinstall script to the installation package. This script will be executed after the payload files have been installed.

With your favored text editor, create a file named postinstall (no file extension) with this content in the scripts sub directory.

#!/bin/sh

export PATH=/usr/bin:/bin:/usr/sbin:/sbin

label="com.scriptingosx.desktopprmanage"
launchdPlist="/Library/LaunchAgents/${label}.plist"

# do not load when target volume ($3) is not current startup volume
if [ "$3" != "/" ]; then
    echo "not installing on startup volume, exiting"
    exit 0
fi

# for LaunchAgent, check if user is logged in
# see https://scriptingosx.com/2019/09/get-current-user-in-shell-scripts-on-macos/

currentUser=$(echo "show State:/Users/ConsoleUser" | scutil | awk '/Name :/ { print $3 }')

# don't load at loginwindow 
if [ "$currentUser" = "loginwindow" ] || [ "$currentUser" = "_mbsetupuser" ]; then
    echo "no user logged in, exiting"
    exit 0
fi

uid=$(id -u "$currentUser")

echo "loading $label"
launchctl bootstrap "gui/$uid" "$launchdPlist"

Set the proper file privileges on the postinstall file with

> chmod 755 scripts/postinstall

First the script sets the PATH. Then some variables for the label and bath to the label. You can change these to adapt the script to other LaunchAgents or LaunchDaemons.

Then the scripts checks the $3 argument. The installer system passes the target volume to the postinstall script in this argument. While it is a not common in deployment workflows any more, you can still install a pkg on a target volume that is not the current boot volume “/“. We remain safe with our script and check for that possibility, and exit without loading the agent.

LaunchAgents have to be loaded or bootstrapped into the current user’s context. The script gets the current user and checks whether the system might be sitting at the login window. We exit without loading if the system is at the login window or if the current user is _mbsetupuser which means the package is being installed while the system is sitting at Setup Assistant during first setup. Now that the LaunchAgent plist is in place, it will be loaded the first time the user logs in.

When there is a user logged in, we bootstrap the LaunchAgent right away. This is where the modern launchctl syntax shows its advantages. With the legacy commands, this would be:

launchctl asuser "$uid" launchctl load "$launchdPlist"

When you load a LaunchDaemon in a postinstall, the script is much simpler, as you don’t need to check for a current user. You can directly load the daemon into the system target domain:

launchctl bootstrap system/ "$launchdPlist"

Unloading before installation

When you are updating existing software with a package, you have to consider the situation where a LaunchAgent or LaunchDaemon is already running. In this case, we should unload before the installation. We can add a preinstall script (also no file extension) to the scripts folder that will be run before the payload is installed. This closely parallels the postinstall script.

#!/bin/sh

export PATH=/usr/bin:/bin:/usr/sbin:/sbin

label="com.scriptingosx.desktopprmanage"
launchdPlist="/Library/LaunchAgents/${label}.plist"

# do not unload when target volume ($3) is not current startup volume
if [ "$3" != "/" ]; then
    echo "not installing on startup volume, exiting"
    exit 0
fi

if ! launchctl list | grep -q "$label"; then
    echo "$label not loaded, exiting"
    exit 0
fi

# for LaunchAgent, check if user is logged in
# see https://scriptingosx.com/2019/09/get-current-user-in-shell-scripts-on-macos/

currentUser=$(echo "show State:/Users/ConsoleUser" | scutil | awk '/Name :/ { print $3 }')

# don't unload at loginwindow or Setup Assistant
if [ "$currentUser" = "loginwindow" ] || [ "$currentUser" = "_mbsetupuser" ]; then
    echo "no user logged in, exiting"
    exit 0
fi

uid=$(id -u "$currentUser")

echo "loading $label"
launchctl bootout "gui/$uid" "$launchdPlist"

Again, set the proper file privileges on the preinstall file with

> chmod 755 scripts/preinstall

There is one extra step compared to the postinstall script. The preinstall uses launchctl list to check if the launchd configuration is loaded, before attempting to unload it.

Building the Package Installer

Now we get to put everything together. You can use pkgbuild to build an installer package:

> pkgbuild --root payload/ --scripts scripts/ --version 1 --identifier com.scriptingosx.desktopprmanage --install-location / DesktopprManage-1.pkg   
pkgbuild: Inferring bundle components from contents of payload/
pkgbuild: Adding top-level preinstall script
pkgbuild: Adding top-level postinstall script
pkgbuild: Wrote package to DesktopprManage-1.pkg

This works well, but it is difficult to memorize all the proper arguments for the pkgbuild command. If you forget or misconfigure one of them it will either fail outright or lead to a pkg installer file with unwanted behavior. I prefer to put the command in a script. Create a script file buildDesktopprManages.sh at the top of your project directory with these contents:

#!/bin/sh

export PATH=/usr/bin:/bin:/usr/sbin:/sbin

pkgname="DesktopprManage"
version="1.1"
identifier="com.scriptingosx.${pkgname}"
install_location="/"
minOSVersion="10.13"

# determine enclosing folder
projectfolder=$(dirname "$0")

# build the component package
pkgbuild --root "${projectfolder}/payload" \
         --identifier "${identifier}" \
         --version "${version}" \
         --ownership recommended \
         --install-location "${install_location}" \
         --scripts "${projectfolder}/scripts" \
         --min-os-version "${minOSVersion}" \
         --compression latest \
         "${pkgname}-${version}.pkg"

This way, you do not have to memorize the pkgbuild command and its arguments and it is easier to modify parameters such as the version. Remember to change the version as you build newer versions of this package going forward!

Some management systems deploy a pkg file using certain MDM commands. Then you need a signed distribution package instead of a simple component pkg file. (You can learn more about the different types of packages in this blog post or this MacDevOps YVR presentation. Building a sign distribution pkg is somewhat more involved. I have a template script for that in the repo.

Conclusion

As I said, building a LaunchAgent for desktoppr manage is not trivial. However, building LaunchAgents and LaunchDaemons and proper installer pkgs to deploy them is an important skill for Mac admins.

There are some other tools available, such as outset, or your management system might have options to run scripts at certain triggers. But in general, I recommend to understand the underlying processes and technologies before using tools, even when the tools are great and a good fit for the task at hand.

You can use this process of building a LaunchAgent and installer pkg for desktoppr as a template for your own scripts and projects. You can find all the files in the desktoppr repo example directory.