Year: 2019

Apple has announced that in macOS 10.15 Catalina the default shell will be zsh.

In this series, I will document my experiences moving bash settings, configurations, and scripts over to zsh.

• Part 1: Moving to zsh
• Part 2: Configuration Files
• Part 3: Shell Options
• Part 4: Aliases and Functions (this article)
• Part 5: Completions
• Part 6: Customizing the zsh Prompt
• Part 7: Miscellanea
• Part 8: Scripting zsh

This series has grown into a book: reworked and expanded with more detail and topics. Like my other books, I plan to update and add to it after release as well, keeping it relevant and useful. You can order it on the Apple Books Store now.

As I have mentioned in the earlier posts, I am aware that there are many solutions out there that give you a pre-configured ‘shortcut’ into lots of zsh goodness. But I am interested in learning this the ‘hard way’ without shortcuts. Call me old-fashioned. (“Uphill! In the snow! Both ways!”)

Aliases

Aliases in zsh work just like aliases in bash. You declare an alias with the alias (built-in) command and it will work as a text replacement at the beginning of the command prompt:

alias ll='ls -al'

You can just copy your alias declarations from your .bash_profile or .bashrc to your .zshrc. I had aliases for .. and cd.. which are now handled by Auto CD and shell correction respectively, so I didn’t bother to move those. (part 3: ‘Shell Options’)

After the alias is declared, you can use it at the beginning of a command. When you try to use the alias anywhere else in the command, the alias will not work:

% sudo ll
sudo: ll: command not found

Global Aliases

This is where zsh has an advantage. You can declare an alias as a ‘global’ alias, and then will be replaced anywhere in the command line:

% alias -g badge='tput bel'
% sudo badge        #<beeps> with privilege

Identifying Aliases

There is one more feature of zsh that is useful with aliases. The which command will show if a command stems from an alias substitution:

% which ll
ll: aliased to ls -l

However, when you try this with global aliases, the substitution occurs before the which command can evaluate the alias, which leads to an unexpected result:

% which badge
/usr/bin/tput
bel not found

You can suppress the alias substitution by escaping the first character or by quoting the entire alias name:

% which \badge
badge: globally aliased to tput bel
% which 'badge'
badge: globally aliased to tput bel

Functions

As with aliases, functions in your zsh configuration will work just as they did in bash.

function vnc() {
    open vnc://"$USER"@"$1"
}

This code in your zsh configuration file will define the vnc function and make it available in the shell.

Autoload Functions

However, zsh has some features which make using functions more flexible. There is (once again) a bit of configuration required to get this working.

Instead of declaring the function directly the configuration file, you can put the function in a separate file. zsh has a built-in variable called fpath which is an array of paths where zsh will look for files defining a function. You can add your own directory to this search path:

fpath+=~/Projects/dotfiles/zshfunctions

Just having a file in the directory is not enough. You still have to tell zsh that you want to use this particular function:

autoload vnc

This command tells zsh: “’Declare a function named vnc. To execute it, load a file named vnc, it is somewhere in the fpath.”

Note: you often see the -U or -Uz option added to the autoload command. These options help avoid conflicts with your personal settings. They suppress alias substitution and ksh-style loading of functions, respectively.

The vnc file in my zshfunctions directory can look like this:

# uses the arguments as hostnames for `open vnc://` (Screen Sharing)
# uses the $USER username as default account name

for x in $@; do 
    open vnc://"$USER"@"$x"
done

The vnc function will open a Screen Sharing session with the current user name pre-filled in.

Initializing Autoload Functions

You could also put the code in the function file into a function block:

function vnc {
    for x in $@; do 
        open vnc://"$vnc_user"@"$x"
    done
}

# initialization code
vnc_user="remote_admin"
alias screen_sharing='vnc'

The function name should match the function name declared with autoload.

When you have additional code outside the function, the autoload behavior changes. When the function is called for the first time, the function will be defined and the code outside the function will be run. The function itself will not be executed on the first run. On subsequent calls, the function will be executed and the code outside the function is ignored.

You can use this to provide setup and initialization code for the function. You can even have more functions defined in the function file. The above example declares and sets a variable to use for account name and an alias for the vnc command.

Since you have to run the function once for the initialization, you often see this syntax in the zsh configuration file:

autoload vnc && vnc

Which means ‘declare the function and if that succeeds run it.’

In some functions, the initialization code will already launch the function itself:

function vnc {
    ...
}
# initialization
vnc_user="remote_admin"
vnc()

Since the behavior will vary from each autoloaded function to the next, be sure to study any documentation or the function’s code.

Identifying Functions

Finally, the which command will show the function code:

 % which vnc
vnc () {
    for x in $@
    do
        open vnc://"$USER"@"$x"
    done
}

The functions command without any parameters, will print all functions (there will be a lot of them). Use functions + to just list the function names.

Debugging Functions

When you are working on complex autoloaded functions, you will at some point have to do some debugging. You can enable tracing for functions with

% functions -t vnc
% vnc Client.local
+vnc:1> x=Client.local
+vnc:2> open vnc://armin@Client.local

You can disable tracing for this function with functions +t vnc.

Next

In the next part we will enable, use and configure tab completions.

Apple has announced that in macOS 10.15 Catalina the default shell will be zsh.

In this series, I will document my experiences moving bash settings, configurations, and scripts over to zsh.

• Part 1: Moving to zsh
• Part 2: Configuration Files
• Part 3: Shell Options (this article)
• Part 4: Aliases and Functions
• Part 5: Completions
• Part 6: Customizing the zsh Prompt
• Part 7: Miscellanea
• Part 8: Scripting zsh

This series has grown into a book: reworked and expanded with more detail and topics. Like my other books, I plan to update and add to it after release as well, keeping it relevant and useful. You can order it on the Apple Books Store now.

Now that we have chosen a file to configure our zsh, we need to decide on ‘what’ to configure and ‘how.’ In this post, I want to talk about zsh’s shell options.

As I have mentioned in the earlier posts, I am aware that there are many solutions out there that give you a pre-configured ‘shortcut’ into lots of zsh goodness. But I am interested in learning this the ‘hard way’ without shortcuts. Call me old-fashioned. (“Uphill! In the snow! Both ways!”)

In the previous post, I listed some features that I would like to transfer from my bash configuration. While researching how to implement these options in zsh, I found a few, new and interesting options in zsh.

The settings from bash which I want in zsh were:

• case-insensitive globbing
• command history, shared across windows and sessions

Note: bash in this series of posts specifically refers to the version of bash that comes with macOS as /bin/bash (v3.2.57).

Note 2: Mono-typed lines starting with a % show commands and results from zsh. Mono-typed lines starting with $ show commands and results in bash

What are Shell Options?

Shell options are preferences for the shell’s behavior. You are using shell options in bash, when you enable ‘trace mode’ for scripts with the set -x command or the bash -x option. (Note: this also works with zsh scripts.)

zsh has a lot of shell options. Many of these options serve the purpose of enabling (or disabling) compatibility with other shells. There are also many options which are specific to zsh.

You can set an option with the setopt command. For compatibility with other shells the setopt command and set -o have the same effect (set an option by name). The following commands set the same option:

set -o AUTO_CD
setopt AUTO_CD

The names or labels of the options are commonly written in all capitals in the documentation but in lowercase when listed with the setopt tool. The labels of the options are case insensitive and any underscores in the label are ignored. So, these commands set the same option:

setopt AUTO_CD
setopt autocd
setopt auto_cd
setopt autoCD

There are quite a few ways to negate or unset an option. First you can use unsetopt or set +o. Alternatively, you can prefix with NO or no to negate an option. The following commands all have the same effect of turning off the previously set option AUTO_CD

unsetopt AUTO_CD
set +o AUTO_CD
unsetopt autocd
setopt NO_AUTO_CD
setopt noautocd

Any options you change will only take effect in the current instance of zsh. When you want to change the settings for all new shells, you have to put the commands in one of the configuration files (usually .zshrc).

Showing the current Options

You can list the existing shell options with the setopt command:

% setopt
combiningchars
interactive
login
monitor
shinstdin
zle

This list only shows options are changed from the default set of options for zsh. These options are marked with <D> (default for all shell emulations) or <Z> (default for zsh) in the documentation or the zshoptions man page.

You can also get a list of all default zsh options with the command:

% emulate -lLR zsh

Some zsh Options I use

As I have mentioned before in my posts on bash configuration, I prefer minimal configuration changes, so I do not feel all awkward and lost when I have to work on an ‘un-configured’ Mac.

These configurations are a personal choice and you should pick and choose your own. You can find a full list of zsh options in the zsh Manual or with man zshoptions.

On the other hand, exploring the options allows us to explore a few useful zsh features.

Case Insensitive Globbing

Note: ‘Globbing’ is a unix/shell term that refers to the expansion of wildcard characters, such as * and ? into full file paths and names. I.e. ~/D* is expanded into /Users/armin/Desktop /Users/armin/Documents /Users/armin/Downloads

Since the file system on macOS is (usually) case-insensitive, I prefer globbing and tab-completion to be case-insensitive as well.

The zsh option which controls this is CASE_GLOB. Since we want globbing to be case-insensitive, we want to turn the option off, so:

setopt NO_CASE_GLOB

You can test this in the shell:

% ls ~/d*<tab>

In zsh tab completion will replace the wildcard with the actual result. So after the tab you will see:

% ls /Users/armin/Desktop /Users/armin/Documents /Users/armin/Downloads

Using tab completion this way to see and possibly edit the actual replacement for wildcards is a useful safety net.

In bash hit the tab key will list possible completions, but not substitute them in the command prompt.

If you do not like this behavior in zsh then you can change to behavior similar to bash with:

setopt GLOB_COMPLETE

Automatic CD

Sometimes you enter the path to a directory, but forget the leading cd:

$ Library/Preferences/
bash: Library/Preferences/: is a directory

% Library/Preferences
zsh: permission denied: Library/Preferences

With AUTO_CD enabled in zsh, the shell will automatically change directory:

% Library/Preferences
% pwd
/Users/armin/Library/Preferences

This works with relative and absolute paths, including the ..:

% ..
% pwd
/Users/armin/Library
% ../Desktop 
% pwd
/Users/armin/Desktop

I have an alias in my .bash_profile that sets the .. command to cd ... Auto CD replaces that functionality and more.

Enable Auto CD with:

setopt AUTO_CD

Shell History

Shells commonly remember previously executed commands and allows you to recall them with the up and down arrow keys, search or special history commands.

Most of those keys work the same in zsh. However, there are a few things you need to configure for zsh history to work as you are used to with bash on macOS.

By default, zsh does not save its history when the shell exits. The history is ‘forgotten’ when you close a Terminal window or tab. To make zsh save its history to a file when it exits, you need to set a variable in the shell:

HISTFILE=${ZDOTDIR:-$HOME}/.zsh_history

Note: this is not a shell option but shell variable or parameter. I will cover some more of those later, You can find a list of variables used by zsh in the documentation.

The HISTFILE variable tells zsh where to store the history data. The syntax ${ZDOTDIR:-$HOME} means it will use the value of ZDOTDIR when it is set or default to the value of HOME otherwise. When a user has set the ZDOTDIR variable to group their configurations files in a specific directory, the history will be stored there as well.

By default zsh simply writes each command in its own line in the history file. You can view the file’s contents with any text editor or list the last few commands:

% tail -n 10 ~/.zsh_history

You can make zsh add a bit more data (timestamp in unix epoch time and elapsed time of the command) by setting the EXTENDED_HISTORY shell option.

setopt EXTENDED_HISTORY

You can set limits on how many commands the shell should remember in the session and in the history file with the HISTSIZE and SAVEHIST variables:

SAVEHIST=5000
HISTSIZE=2000

When the shell reaches this limit the oldest commands will be removed from memory or the history file.

By default, when you exit zsh (for example, by closing the window or tab) this particular instance of zsh will overwrite an existing history file with its history. So when you have multiple Terminal windows or tabs open, they will all overwrite each others’ histories eventually.

You can tell zsh to use a single, shared history file across the sessions and append to it rather than overwrite:

# share history across multiple zsh sessions
setopt SHARE_HISTORY
# append to history
setopt APPEND_HISTORY

Furthermore, you can tell zsh to update the history file after every command, rather than waiting for the shell to exit:

# adds commands as they are typed, not at shell exit
setopt INC_APPEND_HISTORY

When you use a shared history file, it will grow very quickly, and you may want to use some options to clean out duplicates and blanks:

# expire duplicates first
setopt HIST_EXPIRE_DUPS_FIRST 
# do not store duplications
setopt HIST_IGNORE_DUPS
#ignore duplicates when searching
setopt HIST_FIND_NO_DUPS
# removes blank lines from history
setopt HIST_REDUCE_BLANKS

(some of these are redundant)

Most of the time you will access the history with the up arrow key to recall the last command, or maybe a few more steps. You can search through the history with ctrl-R

In zsh, you can also use the !! history substitution, which will be replaced with the entire last command. This is most commonly used in combination with sudo:

% systemsetup -getRemoteLogin
You need administrator access to run this tool... exiting!
% sudo !!
sudo systemsetup -getRemoteLogin
Password:
Remote Login: On

By default, the shell will show the command it is substituting before it is run. But at that point, it is too late to make any changes. When you set the HIST_VERIFY option, zsh will show the substituted command in the prompt instead, giving you a chance to edit or cancel it, or just confirm it.

% systemsetup -getRemoteLogin
You need administrator access to run this tool... exiting!
% sudo !!
% sudo systemsetup -getRemoteLogin
Password:
Remote Login: On

This works for other history substitutions such as !$ or !*, as well. You can find all of zsh’s history expansions in the documentation.

Correction

When you mistype a command or path, the shell is usually unforgiving. In zsh you can enable correction. Then, the shell will make a guess of what you meant to type and ask whether you want do that instead:

% systemprofiler 
zsh: correct 'systemprofiler' to 'system_profiler' [nyae]?

Your options are to

• n: execute as typed
• y: accept and execute the suggested correction
• a: abort and do nothing
• e: return to the prompt to continue editing

I have found this far less annoying and far more useful than I expected. Especially, since it works together with AUTO_CD:

% Dekstop
zsh: correct 'Dekstop' to 'Desktop' [nyae]?

You enable zsh correction with these options:

setopt CORRECT
setopt CORRECT_ALL

Reverting to defaults

Most of the changes mentioned here affect the interactive shell and will have little impact on zsh scripts. However, there are some options that do affect the behavior of things like variable substitutions which will affect scripts.

You can revert the options for the current shell to the default settings with the following command:

emulate -LR zsh

We encountered this command earlier when we listed the default settings. The -l option will list the settings rather than apply them.

If in doubt, it may be useful to add this at the beginning of your zsh scripts.

Next

In the next part we will take a look at aliases and functions.

Apple has announced that in macOS 10.15 Catalina the default shell will be zsh.

In this series, I will document my experiences moving bash settings, configurations, and scripts over to zsh.

• Part 1: Moving to zsh
• Part 2: Configuration Files (this article)
• Part 3: Shell Options
• Part 4: Aliases and Functions
• Part 5: Completions
• Part 6: Customizing the zsh Prompt
• Part 7: Miscellanea
• Part 8: Scripting zsh

This series has grown into a book: reworked and expanded with more detail and topics. Like my other books, I plan to update and add to it after release as well, keeping it relevant and useful. You can order it on the Apple Books Store now.

In part one I talked about Apple’s motivation to switch the default shell and urge existing users to change to zsh.

Since I am new to zsh as well, I am planning to document my process of transferring my personal bash setup and learning the odds and ends of zsh.

Many websites and tutorials leap straight to projects like oh-my-zsh or prezto where you can choose from hundreds of pre-customized and pre-configured themes.

While these projects are very impressive and certainly show off the flexibility and power of zsh customization, I feel this will actually prevent an understanding of how zsh works and how it differs from bash. So, I am planning to build my own configuration ‘by hand’ first.

At first, I actually took a look at my current bash_profile and cleaned it up. There were many aliases and functions which I do not use or broke in some macOS update. I the end, this is what I want to re-create in zsh:

• aliases
  • mostly shortcuts to open files with a specific application
• functions
  • show man pages in a dedicated Terminal window
  • some more simple functions
  • get the frontmost Finder window path
• shell settings
  • case-insensitive globbing
  • case-insensitive path-completion (for bash this is set in .inputrc)
  • command history, shared across windows and sessions
  • use BBEdit as the editor
• prompt:
  • show current working dir
  • show a colored symbol showing the last command’s exit code
  • update the Terminal window title bar to show the cwd

Most of these should be fairly easy to transfer. Some might be… interesting.

But first, where do we put our custom zsh configuration?

zsh Configuration Files

bash has a list of possible files that it tries in predefined order. I have the description in my post on the bash_profile.

zsh also has a list of files it will execute at shell startup. The list of possible files is even longer, but somewhat more ordered.

all users	user	login shell	interactive shell	scripts	Terminal.app
/etc/zshenv	.zshenv	√	√	√	√
/etc/zprofile	.zprofile	√	x	x	√
/etc/zshrc	.zshrc	x	√	x	√
/etc/zlogin	.zlogin	√	x	x	√
/etc/zlogout	.zlogout	√	x	x	√

The files in /etc/ will be launched (when present) for all users. The .z* files only for the individual user.

By default, zsh will look in the root of the home directory for the user .z* files, but this behavior can be changed by setting the ZDOTDIR environment variable to another directory (e.g. ~/.zsh/) where you can then group all user zsh configuration in one place.

On macOS you could set the ZDOTDIR to ~/Documents/zsh/ and then use iCloud syncing (or a different file sync service) to have the same files on all your Macs. (I prefer to use git.)

bash will either use .bash_profile for login shells, or .bashrc for interactive shells. That means, when you want to centralize configuration for all use cases, you need to source your .bashrc from .bash_profile or vice versa.

zsh behaves differently. zsh will run all of these files in the appropriate context (login shell, interactive shell) when they exist.

zsh will start with /etc/zshenv, then the user’s .zshenv. The zshenv files are always used when they exist, even for scripts with the #!/bin/zsh shebang. Since changes applied in the zshenv will affect zsh behavior in all contexts, you should you should be very cautious about changes applied here.

Next, when the shell is a login shell, zsh will run /etc/zprofile and .zprofile. Then for interactive shells /etc/zshrc and .zshrc. Then, again, for login shells /etc/zlogin and .zlogin. Why are there two files for login shells? The zprofile exists as an analog for bash’s and sh’s profile files, and zlogin as an analog for ksh login files.

Finally, there are zlogout files that can be used for cleanup, when a login shell exits. In this case, the user level .zlogout is read first, then the central /etc/zlogout. If the shell is terminated by an external process, these files might not be run.

Apple Provided Configuration Files

macOS Mojave (and earlier versions) includes /etc/zprofile and /etc/zshrc files. Both are very basic.

/etc/zprofile uses /usr/libexec/path_helper to set the default PATH. Then /etc/zshrc enables UTF–8 with setopt combiningchars.

Like /etc/bashrc there is a line in /etc/zshrc that would load /etc/zshrc_Apple_Terminal if it existed. This is interesting as /etc/bashrc_Apple_Terminal contains quite a lot of code to help bash to communicate with the Terminal application. In particular bash will send a signal to the Terminal on every new prompt to update the path and icon displayed in the Terminal window title bar, and provides other code relevant for saving and restoring Terminal sessions between application restarts.

However, there is no /etc/zshrc_Apple_Terminal and we will have to provide some of this functionality ourselves.

Note: As of this writing, /etc/zshrc in the macOS Catalina beta is different from the Mojave /etc/zshrc and provides more configuration. However, since Catalina is still beta, I will focus these articles on Mojave and earlier. Once Catalina is released, I may update these articles or write a new one for Catalina, if necessary.

Which File to use?

When you want to use the ZDOTDIR variable to change the location of the other zsh configuration files, setting that variable in ~/.zshenv seems like a good choice. Other than that, you probably want to avoid using the zshenv files, since it will change settings for all invocations of zsh, including scripts.

macOS Terminal considers every new shell to be a login shell and an interactive shell. So, in Terminal a new zsh will potentially run all configuration files.

For simplicity’s sake, you should use just one file. The common choice is .zshrc.

Most tools you can download to configure zsh, such as ‘prezto’ or ‘oh-my-zsh’, will override or re-configure your .zshrc. You could consider moving your code to .zlogin instead. Since .zlogin is sourced after .zshrc it can override settings from .zshrc. However, .zlogin is only called for login shells.

The most common situation where you do not get a login shell with macOS Terminal, is when you switch to zsh from another shell by typing the zsh command.

I would recommend to put your configuration in your .zshrc file and if you want to use any of the theme projects, read and follow their instructions closely as to how you can preserve your configurations together with theirs.

Managing the shell for Administrators

MacAdmins may have the need to manage certain shell settings for their users, usually environment variables to configure certain command line tool’s behaviors.

The most common need is to expand the PATH environment variable for third party tools. Often the third party tools in question will have elaborate postinstall scripts that attempt to modify the current user’s .bash_profile or .bashrc. Sometimes, these tools even consider that a user might have changed the default shell to something other than bash.

On macOS, system wide changes to the PATH should be done by adding files to /etc/paths.d.

As an administrator you should be on the lookout for scripts and installers that attempt to modify configuration files on the user level, disable the scripts during deployment, and manage the required changes centrally. This will allow you to keep control of the settings even as tools change, are added or removed from the system, while preserving the user’s custom configurations.

To manage environment variables other than PATH centrally, administrators should consider /etc/zshenv or adding to the existing /etc/zshrc. In these cases you should always monitor whether updates to macOS overwrite or change these files with new, modified files of their own.

Summary

There are many possible files where the zsh can load user configuration. You should use ~/.zshrc for your personal configurations.

There are many tools and projects out there that will configure zsh for you. This is fine, but might keep you from really understanding how things work.

MacAdmins who need to manage these settings centrally, should use /etc/paths.d and similar technologies or consider /etc/zshenv or /etc/zshrc.

Apple’s built-in support for zsh in Terminal is not as detailed as it is for bash.

Next: Part 3 – Shell Options