Terminal Primer – Part 3 – Special Characters

It is vacation time here in the Scripting OS X headquarters and I will be traveling with family for most of August. The Weekly News Summaries are on also on vacation, but to compensate for that I will be posting a draft of parts of my next book on macOS Terminal and bash. Let me know how you like them or if you think something important is wrong or missing. You can give feedback in the comments, over Twitter or in the MacAdmins forum (also scriptingosx). Thank you for your interest and feedback!

There are a group of characters that have special meaning in bash.

| & : ; ( ) < > ~ * @ ? ! $ # [ ] { } \ / ' " `

Also whitespace characters also need to be treated with care:

space, tab, return, and newline

The space character is a legal and often used character in file names on macOS. However, in bash and other shell commands , a space character (and other whitespace characters) separates a command from an argument and the arguments from each other.

When you try to enter a filename with a space, you will get an error:

$ cd /Library/Application Support
-bash: cd: /Library/Application: No such file or directory

To convince the shell that ‘/Library/Application Support’ belongs together, you can either ‘escape’ the space character or ‘quote’ the path.

Experienced users who have worked in a UNIX environment for a long time tend to avoid these special characters in filenames. However, as a system administrator, your users will probably not heed any rules you may want to impose. You will have to deal with many possible combinations.

Escaping Characters

The escape character in bash is the backslash \. A character that follows a backslash will be treated with no special meaning:

$ cd /Library/Application\ Support

In Finder, you can name files and folders nearly any way you want. When you encounter special characters from the list above you have to escape them with backslash. For a directory named ‘Project (Old & New)’ you would type:

$ cd Project\ \(Old\ \&\ New\)

All of this further confused by the fact that the shell will happily display the path with the unescaped special characters:

$ pwd
/Users/armin/Project (Old & New)

Separation Characters

In bash (and in Unix in general) files and directory names cannot contain a forward slash / since the character is used in paths to separate directories. However, Finder lets you name a file or folder with a forward slash, e.g. ‘Yes/No’.

On the other hand, Finder does not let you name a file or folder with a colon :. The underlying HFS+ file system uses the colon as a separator.

This conflict is solved by translating a / in the Finder (and underlying file system) to a colon : in the shell and vice versa.

A folder named ‘Yes/No/Maybe’ in Finder will appear as Yes:No:Maybe in the shell and you have to escape the colons when using the path in a command:

$ cd Yes\:No\:Maybe

Note: some characters that are legal on macOS might not be on file servers, which are usually hosted by other operating systems.

Quoting

As seen above, escaping characters can make the path quite unreadable. You can also place the name or path in quotes:

$ cd 'Project (Old & New)'

In bash you can use single quotes ' or double quotes " to quote paths.

Single quotes are more effective. Any character in single quotes is used as is, with no special function. Even the backslash character has no special function. The only character you cannot use in single quotes is the single quote itself.

Double quotes " are ‘weaker’ quoting. Double quotes remove the special function from all special characters except $, `, \, and !. Within double quotes you can use the backslash to escape $, ", `, and \ (but not the !).

Escape Strategies

In general, single quotes are most useful and easiest to use. However, you cannot use single quotes when the filename contains a single quote.

Double quotes still require some characters to be escaped with the backslash and cannot deal with an exclamation mark !.

Backslash escaping works in nearly all cases, but can be tricky to type right and is quite illegible.

name (in Finder) Backslash Escape single Quotes Double Quotes
My Great Folder My\ Great\ Folder 'My Great Folder' "My Great Folder"
“New” Files \"New\"\ Files '"New" Folder' "\"New\" Folder"
‘Old’ Stuff \'Old\'\ Stuff cannot escape ' "'Old' Stuff"
Important! Important\! 'Important!' cannot escape !
Bump \m/ Bump \\m\: 'Bump \m:' "Bump \\m:"
Do@Home Do\@Home 'Do@Home' "Do@Home"
Yes/No/Maybe Yes\:No\:Maybe 'Yes:No:Maybe' "Yes:No:Maybe"
Project (Old & New) Project\ \(Old\ \&\ New\) 'Project (Old & New)' "Project (Old & New)"
Profit$$$ Profit\$\$\$ 'Profit$$$' "Profit\$\$\$"

Quoting and Tab Completion

When typing paths, always use tab completion to be safe. Tab completion uses backslash escaping by default.

$ cd Proj⇥
$ cd Project\ \(Old\ \&\ New\)/

However, when you start a quoted path, tab completion will complete in quoted form.

$ cd 'Pro⇥
$ cd 'Projects (Old & New)'/

Tab completion is even smart enough to change the approach when the strategy you chose (i.e. double quotes) cannot work:

$ cd "Imp⇥
$ cd Important\!/

Quoting and Home Path

Since you generally use quoting to avoid bash changing characters, you cannot use the ~ to get a short cut to your home directory in quotes.

However, you can leave the ~ outside of the quotes and get the best of both worlds:

$ cd ~/'Project (Old & New)'

When you use double quotes, you can also use the $HOME environment variable:

$ cd "$HOME/Project (Old & New)"

Next: Commands

One thought on “Terminal Primer – Part 3 – Special Characters”

Leave a Reply

Your email address will not be published. Required fields are marked *