Where PATHs come from

In an earlier post we talked about how to append to the PATH variable, so you can add your own directories to bash’s search path.

In macOS the default PATH on a ‘clean’ installation is:

$ echo $PATH
/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin:

However, if you have installed some tools (such as the macOS Server.app, Xquartz or Munki) you will see those in the PATH as well:

$ echo $PATH
/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin:/opt/X11/bin:/Applications/Server.app/Contents/ServerRoot/usr/bin:/Applications/Server.app/Contents/ServerRoot/usr/sbin:/usr/local/munki:

Where does this pre-set PATH come from?

Since the PATH is pre-set on a clean new account without a .bashrc or .bash_profile, we have to look in a central location which applies to all users. In another earlier post, we saw that /etc/profile is run for every user shell, even before a .bash_profile is executed. When you look into this file, you see that the very first set of commands look like this:

if [ -x /usr/libexec/path_helper ]; then
    eval `/usr/libexec/path_helper -s`
fi

This looks very promising. The path_helper tool has a man page. This tool does a few things to assemble the PATH. First it reads the file /etc/paths which on macOS looks like this:

/usr/local/bin
/usr/bin
/bin
/usr/sbin
/sbin

So this is where the default ‘clean’ macOS PATH comes from. Then path_helper will read every file from /etc/paths.d and append each line of each file in that directory to the PATH as well. This is where optional and third party applications and tools, like Xquartz or Munki, can install their own additions to the PATH for all users.

(Files in this folder will be read in alpha-numerical order of the filename. Some tools, like Xquartz, attempt to influence the order by preprending a number, e.g. 40-XQuartz.)

Finally, if path_helper runs in an environment where PATH is already set, it will append that PATH value to what it built from the files and then remove duplicates.

path_helper does not change the environment variable directly, but it generates the commands necessary to set the PATH correctly. It will generate the right commands wether it is called from a bourne type shell (on macOS: sh, bash, ksh and zsh) or a csh type shell (on macOS: csh and tcsh). You can see the output of the two styles by running path_helper with the -s or -c options:

$ /usr/libexec/path_helper -s
PATH="/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin"; export PATH;
$ /usr/libexec/path_helper -c
setenv PATH "/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin";

(You have to type the full path to path_helper because, ironically, but intentionally, /usr/libexec is not in the standard PATH.)

To actually execute the commands generated by path_helper you can use the eval command, like the /etc/profile does:

$ eval $(/usr/libexec/path_helper)

Don’t touch my Profile!

Some command line tool installers understandably feel the need to add their tools to the default PATH. Since there is no unified approach among different flavors of UNIX and Linux on how to do this, you will find several different approaches. Some tools will edit /etc/profile and others will look for the various profile files in a user’s home directory and edit those. Usually the installation process will append a line that appends their tools directory to the PATH.

One example for this is the Python 3 installer. It contains a compnent package that will attempt to determine which profile file you are using and appends a line to append to the PATH.

However, this is not only highly intrusive but also quite fragile. Changes to /etc/profile might be overwritten by a future macOS update. Changes to a user’s profile file, might be overwritten by the user. Also the installer will only append their setting to the current user, not other users that may be present or created in the system.

Sample paths.d installer package

On the other hand, dropping a file into /etc/paths.d with a package installer will affect all users on a system. The file in paths.d can be updated for future updates if necessary and is also easily identified and removed by an admin. It will work wether it is installed to the startup volume or another volume. It can be pushed with management tools.

Building an installer for a file in /etc/paths.d is very simple:

$ mkdir CustomToolPathInstaller
$ cd CustomToolPathInstaller
$ mkdir payload
$ echo "/usr/local/customtool" >> payload/customtool
$ pkgbuild --root payload --install-location /private/etc/paths.d --version 1.0  --identifier com.example.customtool.path CustomToolPath.pkg
pkgbuild: Inferring bundle components from contents of payload
pkgbuild: Wrote package to CustomToolPath.pkg

Only five commands, three of which create the folder structure. You can find this sample project (which is slightly more elaborate) on my GitHub.

If you want to learn more about building installer packages for macOS, please read my book “Packaging for Apple Administrators”.

What about MANPATH?

This is usually not used on macOS since the the default settings for the man tool are quite flexibel. (Look at the man page for man and the file /etc/man.conf for details.) However, if a MANPATH environment variable is set when path_helper runs, it will also assemble the command to set the MANPATH built in a similar way to the PATH from the files /etc/manpaths and the directory /etc/manpaths.d.

Usually the MANPATH is not set on macOS so you will not see this. But if you want to manage your MANPATH and want to leverage path_helper all you have to do is set the MANPATH.

$ export MANPATH=/usr/share/man
$ /usr/libexec/path_helper
PATH="/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin"; export PATH;
MANPATH="/usr/share/man:/usr/local/share/man"; export MANPATH;

Re-order the PATH

We have seen path_helper is extremely useful. There is one caveat, however. path_helper may reorder your PATH. Imagine you are pre-pending ~/bin to your PATH because you want to override some standard tools with your own. (Dangerous, but let’s assume you know what you are doing.) Then some process launches a subshell which can call path_helper again. path_helper will ‘find’ your additions to the defined PATH, but it will append to the list of default paths from /etc/paths and /etc/paths.d, changing your order and thus which tools will be used.

$ export PATH=~/bin:$PATH
$ echo $PATH
/Users/armin/bin:/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin
$ /usr/libexec/path_helper 
PATH="/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin:/Users/armin/bin"; export PATH;

You can see behavior like this when you use Xterm (The X11 based terminal in Xquartz) which does not execute .bash_profile but still picks up the PATHenvironment variable from somewhere…

# in Xquartz Terminal:
bash-3.2$ echo $PATH
/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin:/opt/X11/bin:/Applications/Server.app/Contents/ServerRoot/usr/bin:/Applications/Server.app/Contents/ServerRoot/usr/sbin:/usr/local/munki:/Library/Frameworks/Python.framework/Versions/3.5/bin:/Users/armin/bin:/opt/X11/bin

A better way to override built-in commands which is not affected by path_helper would be to use bash aliases or functions in your profile.