Category: Tools

In an earlier post we talked about how to append to the PATH variable, so you can add your own directories to bash’s search path.

In macOS the default PATH on a ‘clean’ installation is:

$ echo $PATH
/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin:

However, if you have installed some tools (such as the macOS Server.app, Xquartz or Munki) you will see those in the PATH as well:

$ echo $PATH
/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin:/opt/X11/bin:/Applications/Server.app/Contents/ServerRoot/usr/bin:/Applications/Server.app/Contents/ServerRoot/usr/sbin:/usr/local/munki:

Where does this pre-set PATH come from?

Since the PATH is pre-set on a clean new account without a .bashrc or .bash_profile, we have to look in a central location which applies to all users. In another earlier post, we saw that /etc/profile is run for every user shell, even before a .bash_profile is executed. When you look into this file, you see that the very first set of commands look like this:

if [ -x /usr/libexec/path_helper ]; then
    eval `/usr/libexec/path_helper -s`
fi

This looks very promising. The path_helper tool has a man page. This tool does a few things to assemble the PATH. First it reads the file /etc/paths which on macOS looks like this:

/usr/local/bin
/usr/bin
/bin
/usr/sbin
/sbin

So this is where the default ‘clean’ macOS PATH comes from. Then path_helper will read every file from /etc/paths.d and append each line of each file in that directory to the PATH as well. This is where optional and third party applications and tools, like Xquartz or Munki, can install their own additions to the PATH for all users.

(Files in this folder will be read in alpha-numerical order of the filename. Some tools, like Xquartz, attempt to influence the order by preprending a number, e.g. 40-XQuartz.)

Finally, if path_helper runs in an environment where PATH is already set, it will append that PATH value to what it built from the files and then remove duplicates.

path_helper does not change the environment variable directly, but it generates the commands necessary to set the PATH correctly. It will generate the right commands wether it is called from a bourne type shell (on macOS: sh, bash, ksh and zsh) or a csh type shell (on macOS: csh and tcsh). You can see the output of the two styles by running path_helper with the -s or -c options:

$ /usr/libexec/path_helper -s
PATH="/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin"; export PATH;
$ /usr/libexec/path_helper -c
setenv PATH "/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin";

(You have to type the full path to path_helper because, ironically, but intentionally, /usr/libexec is not in the standard PATH.)

To actually execute the commands generated by path_helper you can use the eval command, like the /etc/profile does:

$ eval $(/usr/libexec/path_helper)

Don’t touch my Profile!

Some command line tool installers understandably feel the need to add their tools to the default PATH. Since there is no unified approach among different flavors of UNIX and Linux on how to do this, you will find several different approaches. Some tools will edit /etc/profile and others will look for the various profile files in a user’s home directory and edit those. Usually the installation process will append a line that appends their tools directory to the PATH.

One example for this is the Python 3 installer. It contains a compnent package that will attempt to determine which profile file you are using and appends a line to append to the PATH.

However, this is not only highly intrusive but also quite fragile. Changes to /etc/profile might be overwritten by a future macOS update. Changes to a user’s profile file, might be overwritten by the user. Also the installer will only append their setting to the current user, not other users that may be present or created in the system.

Sample paths.d installer package

On the other hand, dropping a file into /etc/paths.d with a package installer will affect all users on a system. The file in paths.d can be updated for future updates if necessary and is also easily identified and removed by an admin. It will work wether it is installed to the startup volume or another volume. It can be pushed with management tools.

Building an installer for a file in /etc/paths.d is very simple:

$ mkdir CustomToolPathInstaller
$ cd CustomToolPathInstaller
$ mkdir payload
$ echo "/usr/local/customtool" >> payload/customtool
$ pkgbuild --root payload --install-location /private/etc/paths.d --version 1.0  --identifier com.example.customtool.path CustomToolPath.pkg
pkgbuild: Inferring bundle components from contents of payload
pkgbuild: Wrote package to CustomToolPath.pkg

Only five commands, three of which create the folder structure. You can find this sample project (which is slightly more elaborate) on my GitHub.

If you want to learn more about building installer packages for macOS, please read my book “Packaging for Apple Administrators”.

What about MANPATH?

This is usually not used on macOS since the the default settings for the man tool are quite flexibel. (Look at the man page for man and the file /etc/man.conf for details.) However, if a MANPATH environment variable is set when path_helper runs, it will also assemble the command to set the MANPATH built in a similar way to the PATH from the files /etc/manpaths and the directory /etc/manpaths.d.

Usually the MANPATH is not set on macOS so you will not see this. But if you want to manage your MANPATH and want to leverage path_helper all you have to do is set the MANPATH.

$ export MANPATH=/usr/share/man
$ /usr/libexec/path_helper
PATH="/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin"; export PATH;
MANPATH="/usr/share/man:/usr/local/share/man"; export MANPATH;

Re-order the PATH

We have seen path_helper is extremely useful. There is one caveat, however. path_helper may reorder your PATH. Imagine you are pre-pending ~/bin to your PATH because you want to override some standard tools with your own. (Dangerous, but let’s assume you know what you are doing.) Then some process launches a subshell which can call path_helper again. path_helper will ‘find’ your additions to the defined PATH, but it will append to the list of default paths from /etc/paths and /etc/paths.d, changing your order and thus which tools will be used.

$ export PATH=~/bin:$PATH
$ echo $PATH
/Users/armin/bin:/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin
$ /usr/libexec/path_helper 
PATH="/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin:/Users/armin/bin"; export PATH;

You can see behavior like this when you use Xterm (The X11 based terminal in Xquartz) which does not execute .bash_profile but still picks up the PATHenvironment variable from somewhere…

# in Xquartz Terminal:
bash-3.2$ echo $PATH
/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin:/opt/X11/bin:/Applications/Server.app/Contents/ServerRoot/usr/bin:/Applications/Server.app/Contents/ServerRoot/usr/sbin:/usr/local/munki:/Library/Frameworks/Python.framework/Versions/3.5/bin:/Users/armin/bin:/opt/X11/bin

A better way to override built-in commands which is not affected by path_helper would be to use bash aliases or functions in your profile.

Mac Admins have to work a lot in Terminal. This seems counter-intuitive for an OS that is famed for its user interaction. I can’t talk for all admins, but for me the strength of macOS/OS X was always in the combination of ‘clicky’ UI and command line. When you know what you are doing, you can get the best of both worlds.

I remember an Apple marketing slogan: “The power of Unix, the simplicity of Mac” This is from OS X Lion, so more than five years old by now. The future will show how long Apple still values the ‘powerful’ Unix underpinnings. But for now, they are still available and I am going to use them.

All of that said, the CLI and the UI are not entirely separate areas in macOS, there is a lot of overlap and there are functions in Finder and Terminal that allow for quick interaction between the two.

Finder to Terminal

You can drag any folder from Finder to the Terminal application icon in the dock and Terminal will open a new window and change the working directory to the folder you dragged.

Movie 1: Drag Folder onto Terminal

You can drag the folder icon from the Finder window title bar, as well.

Movie 2: Drag Folder from Title Bar onto Terminal

When you drag any file into an open Terminal window, it will insert the full path to that file or folder with spaces and other special characters properly escaped.

Movie 3: Drag File onto Terminal

You can drag multiple files and Terminal will insert all of their paths, separated by spaces. For example you can type file[space] in Terminal and then drag multiple files into that window and hit return, to get information on the file type.

Movie 4: Drag Multiple Files onto Terminal

If you prefer, you can get the same effect with copy and paste. Just select the files in Finder, choose ‘Copy’ (⌘C), switch to Terminal and ‘Paste’ (⌘V).

Update: I knew I had forgotten one. Thanks to Elliot Jordan who pointed this one out on Twitter:

Dragging a folder into a Terminal window while holding the command (⌘) key will add cd before the path to a folder. When you command-drag a file it will cd to the enclosing folder of that file.

Movie 5: Command Drag a Folder to Terminal

Getting Finder path from Terminal

If you are already in Terminal and want to get the frontmost Finder window, we have to do some homework first. (I got the idea for this script from this post, though I have modified its solution somewhat.) This command

$ osascript -e 'tell app "Finder" to get posix path of ((target of window 1) as alias)'

will give us the correct path, but it has two downsides: a) it is awfully complex to type repeatedly and b) it fails with an error if no Finder window is open.

To avoid typing this long command every time, we have two options. You can either define the command as a function in your .bash_profile (or the respective profile for your preferred shell) or you can save it as a script in your $PATH.

To define it as a function, add this to the .bash_profile:

# prints the path of the front Finder window. Desktop if no window open
function pwdf () {
    osascript <<EOS
        tell application "Finder"
            if (count of Finder windows) is 0 then
                set dir to (desktop as alias)
            else
                set dir to ((target of Finder window 1) as alias)
            end if
            return POSIX path of dir
        end tell
EOS
}

# changes directory to frontmost Finder window
alias cdf='pwdf; cd "$(pwdf)"'

The pwdf function will just print the path to the frontmost Finder window to stdout.

There is also added an alias to quickly change directories to the frontmost Finder window. cdf will also print the path it is changing to, like the cd - command. (Which changes to the previous working directory.) This has to be defined as an alias, since scripts cannot change a shell’s working directory.

If you prefer to define the pwdf command from a script file use the following code:

Save this as a text file without extension in a folder in your $PATH, and set the executable bit with chmod +x /path/to/pwdf. You also need to remove (or comment) the function from your profile, since that would override the script.

Using the script form of osascript allows us to pass arguments into the AppleScript. (You can do that with function as well, but the syntax gets really messy quickly.) This script will list the paths to all open Finder windows with the -a|--all argument. Also when you provide a string as an argument it will search for a Finder window containing that string:

$ pwdf Pref
/Users/armin/Library/Preferences/

The open command

You can also go from Terminal to the Finder. This usually as simple as typing

$ open .

This will open the current working directory ‘.’ in a Finder window.

This is usually where most online ‘hints’ for the open command start and end. However, open has so much more to offer. So much more, in fact, that I will cover the open command in a separate article.

I stumbled over these option this morning. I do not know when they were introduced, but I can see the options in 10.11 and 10.12. You can see them yourself with plutil -help. (The options are not listed in the man page.)

Note: Managing and editing Property List files and preferences is covered in much more detail and depth in my book “Property Lists, Preferences and Profiles for Apple Administrators“

Quick recap: plutil manipulates property list files. Its main use up to now was to convert between property list formats (mainly from binary plists to something readable)

$ plutil -convert xml1 /path/to/propertylist.plist

and to check wether the syntax is valid

$plutil -lint /path/to/propertylist.plist

On Sierra, when you run plutil -help you see some new options. These allow you to directly manipulate keys and values in a property list. This may be useful to replace PListBuddy and defaults to manipulate property lists.

When testing this I noticed one downside of plutil immediately: it cannot be used to create a new property list file. Copy this to create an empty plist file:

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict/>
</plist>

Inserting a new key/value pair

$ plutil -insert somekey -string somevalue test.plist
$ plutil -insert flag -bool YES test.plist
$ plutil -insert bestNumber -integer 1 test.plist
$ plutil -insert pi -float 3.141592 test.plist

This is pretty straightforward.

Inspecting a property list

You can use the -p option to check our progress:

$ plutil -p test.plist
{
  "newkey" => "newvalue"
  "pi" => 3.141592
  "bestNumber" => 1
  "flag" => 1
}

This uses a non-standard output format, and the help text warns to use this parse plists. But it will do to look at the content.

Note: you can use plutil -p to read the content of binary plists without converting!

Modifying values

You can modify values with the -replace option.

$ plutil -replace flag -bool NO test.plist

Note that you can create new entries with -replace:

$ plutil -replace otherkey -string othervalue test.plist

But you cannot overwrite an existing value with -insert.

Deleting values

Very straightforward:

$ plutil -remove otherkey test.plist

Arrays

You can insert an empty:

$ plutil -insert list -xml '<array/>' test.plist

or

$ plutil -insert list -json '[]' test.plist

and add items to the array

$ plutil -insert list.0 -string 'list item 1' test.plist
$ plutil -insert list.1 -string 'list item 2' test.plist
$ plutil -insert list.2 -string 'list item 3' test.plist

or do it all at once

$ plutil -replace list -json '[ "yes", "no", "maybe" ]' test.plist

Dictionaries

$ plutil -insert dictionary -xml '<dict/>' test.plist
$ plutil -replace dictionary -json '{}' test.plist
$ plutil -insert dictionary.key1 -string value1 test.plist
$ plutil -replace dictionary -json '{ "otherkey" : "othervalue" }' test.plist

Getting Values

It looks like -extract is meant to get values from a property list, but there is caveat. -extract will not merely get the value of a key in the property list but will write it to a new file! And by default if you do not provide an new output file path with the -o option it will overwrite the current file with the extracted data.

The proper, non-destructive syntax to use -extract is:

$ plutil -extract list xml1 -o - test.plist 
$ plutil -extract list json -o - test.plist 

This will print a full property list file to stdout. The -o - option tells plutil to print to stdout. You can give a filename instead of the -.

Since the output is encumbered with the json or xml syntax, it will be hard to use this to get to property list values in shell scripts. However, it still may be useful to, well, extract property list data from a complex plist file.

Conclusion

Keep in mind that there now is an alternative to defaults and PlistBuddy. Not having to convert a plist before changing data might be helpful, as well as the possibility to manipulate arrays and dictionaries with key paths. (You still should always use defaults when working with preference plist files, since defaults will go through the preferences system and possibility notify a process to update data.)

If you are using python or a similar high level scripting language it will still be more effective to use the libraries for property lists.