SSH Keys, Part 2: Client Verification

So far in this series of posts on ssh on macOS:

Please consider supporting Scripting OS X by buying one of my books!

In the previous post we discussed how ssh uses public and private keys to secure the data sent back and forth, between the remote computer (host) and your machine (client). The keys allow you to be sure that you are talking to the correct host and not an imposter.

However, how does the host know that you are who you claim to be?

Open Sesame…

The default way to authenticate yourself to the ssh host is to give it the correct username and password combination. This is and has been a common practice for computers, servers, websites for a long time and it works for ssh as well.

You will obviously need an account on the host for this to work. In larger organizations the computers will often be connected or bound to a directory service, to centralize the user and group management. If the computers are not bound to a central directory, as is often the case with mobile computers, you can use your management system to install one or more managed users on the system.

In the Sharing Preference Pane, you can control which users or groups of users can use ssh or ‘Remote Login’ to connect to this Mac. Behind the scenes the users and groups that can connect with ssh are in the group. You can find some scripts to manage the controls in this post.

Keys for the Client

Passwords, and user access control groups already provide a strong security, but have some disadvantages as well:

  • you have to re-enter the password on every connection. This is especially tedious when you send individual commands with the ssh hostname command syntax
  • automation is insecure because you have to provide the password in the script (in clear text)

With ssh we can use the same asymmetric keys to validate the client to the host, as we did to verify the host.

Sidenote: you will see a lot of actual keys and fingerprints in this post. While sharing the public key should not be a security issue, I deleted them all after finishing this post, just to be safe.

Creating a Key

First, you need to generate the key for your user. This is simple enough with ssh-keygen:

$ ssh-keygen -t rsa -b 2048 -C "Armin RSA 2017-07-06"
Generating public/private rsa key pair.
Enter file in which to save the key (/Users/armin/.ssh/id_rsa): 
Enter passphrase (empty for no passphrase): 
Enter same passphrase again: 
Your identification has been saved in /Users/armin/.ssh/id_rsa.
Your public key has been saved in /Users/armin/.ssh/
The key fingerprint is:
SHA256:jRydlFk3rRTpuKshFvnzon7UnzcYiHLQhKpQvLqPrl0 Armin RSA 2017-07-06
The key's randomart image is:
+---[RSA 2048]----+
|  .     . .+. ++ |
|   o   . +o. .o..|
|  . . . + o  + . |
| . . . o *  . o  |
|  o .   S o...   |
| . .   . =..o.   |
|  . E   =.+  oo. |
| o..   . .o+..o..|
|+oo.   .oo.o.  ..|

This command will create a new key pair of type ‘rsa’ (-t rsa) with a bit length of 2048 (-b 2048) and a comment of Armin RSA 2017-07-06.

RSA is the recommended type for best cross platform compatibility. Mac hosts and clients also support ECDSA, and ED25519. (DSA is in the list but considered insecure and obsolete.) When you connect to other, non-macOS, hosts, ask the administrators for the best key type to use.

The bit length of the key determines how complex the key is. Longer bit lengths are more secure, but require more CPU power for the encryption and decryption. Different key types can have different values for the bit length, read the ssh-keygen man page for details.

The comment is optional, but provides a way to add a label to a key which will help you identify it later. I usually add who created and when and if the key is for a specific purpose.

Next the ssh-keygen tool will ask where to store the key pair. Since this will be our ‘generic’ key, the default name and location ~/.ssh/id_rsa (or id_ecdsa, etc.) is exactly what we want. Sometimes you may have to create a key specifically for a single host, then you can choose a different name here, so you do not overwrite your other keys.

Next the tool will ask you for a passphrase. This will be used to encrypt (lock) the private key, so even if someone gets their hands on the private key file, they will not be able to use it. Since the private key represents your ssh identity it should be protected well. Use a password generator to create a long passphrase. Store the passphrase in a secure place (Password manager, you will need it again) and enter it here (twice).

Finally, the tool will show the files it generated (the private and public key), its fingerprint (with the comment) and a ascii art representation of the key, which is supposed to be more visually recognizable than the alphanumeric fingerprint.

Copying the Key to the Host

When we set up an ssh connection to a host we need to copy its public key into our ssh settings (the known_hosts file). Similarly we need to copy our public key to the host.

You have to copy the public key (i.e. to the host and append its contents to the ~/.ssh/authorized_keys on the host in your user home directory. On some systems, the file may also be called authorized_keys2 and/or be in a different location.

On Mac OS X versions 10.11 and older you can use this command:

$ cat ~/.ssh/ | ssh [user@] "cat >> ~/.ssh/authorized_keys"

This will pipe the contents of your public key file into ssh, which sends it to the remote host and appends (>>) it to the ~/.ssh/authorized_keys file. (This will fail if the ~/.ssh directory does not exist yet on the host.)

On many Linux systems and on macOS Sierra 10.12 there is a tool to help with this. You can use

$ ssh-copy-id [user@]

By default, the ssh-copy-id command will copy all available public keys to the host. You can tell it to use a specific key with the -i option. Either way, since the key is not yet in place, the command will need to provide you with the host’s password to authenticate to the host.

The ssh-copy-id command gives helpful additional information:

$ ssh-copy-id
/usr/bin/ssh-copy-id: INFO: Source of key(s) to be installed: "/Users/armin/.ssh/"
/usr/bin/ssh-copy-id: INFO: attempting to log in with the new key(s), to filter out any that are already installed
/usr/bin/ssh-copy-id: INFO: 1 key(s) remain to be installed -- if you are prompted now it is to install the new keys

Number of key(s) added:        1

Now try logging into the machine, with:   "ssh ''"
and check to make sure that only the key(s) you wanted were added.

Now, when you attempt to connect to the remote host, you will be prompted to unlock your private key:

$ ssh
Enter passphrase for key '/Users/armin/.ssh/id_ecdsa': 
Last login: Thu Jul  6 09:00:28 2017 from [IP Address]
mac:~ armin$

and when you look at the contents of ~/.ssh/authorized_keys you will see your public key listed:

mac:~ armin$ more .ssh/authorized_keys 
ecdsa-sha2-nistp521 AAAAE2VjZHNhLXNoYTItbmlzdHA1MjEAAAAIbmlzdHA1MjEAAACFBAC3TR7ZYSjJO01JVNYMeXWNNgglNulQVuGGTnE2JiBULbs9In9uWrFWyqOXodi4XCs7lY92undONnN4wQKumsN/RgBnEDATKZZdh5OPI72ZGrmnPNXVUbU8mjzIJ4LpWfTu0x6nqR9zBu1LPNW9xPoQNwTogh7jIHxvEcYav1slKjeRkg== Armin ECDSA Key 2017-07-06

Note that the comment we added when creating the key is part of the public key (and its fingerprint). Considering that this file can contain multiple keys from different sources, the comment will help a lot in locating the proper key.

When a key is compromised or just not used any more, it should be removed from the host’s authorized_keys file. This file should be audited frequently.

Agent of Keys

When we connected to the remote host after copying the key, we did not have to enter the password for the host, because the key pair confirmed our identity. However, you still had to unlock the private key with the passphrase. You stored that somewhere secure, right? Without the passphrase the key is useless.

This does simplify passwords, because you can use the same key for multiple hosts, and you don’t have to memorize all those passwords. However, entering the passphrase over and over is tedious and error prone, too. Thankfully there is a solution to this. There is a process that can store the passphrase for the duration of your login session. The process is called ssh-agent. However you do not talk directly with this tool but use ssh-add to manage it instead. And on macOS you don’t even have to do that, because ssh-add and ssh-agent are integrated with the macOS Keychain.

The role of the ssh-agent is to hold on to your private key passphrases for the duration of your login session. On other systems (or with Keychain integration disabled) you have to add the passphrase once per login session with

$ ssh-add ~/.ssh/id_rsa

(this will prompt for the passphrase)

You can list the passphrases that ssh-agent is holding with

$ ssh-add -l

and clear all of them with

 $ ssh-add -D

Keychain Integration

Note: on macOS Sierra (10.12), Keychain integration is not enabled by default. To enable it, add these two lines at the top of ~/.ssh/config (create the file if necessary):

AddKeysToAgent yes
UseKeychain yes

With Keychain integration enabled, ssh-agent will also look for passphrases stored in the keychain and then remember them for the rest of the session. The passphrase is stored as an application password with the path to the private key in the name. So as long as the keychain is unlocked, it will find the passphrase in the keychain and use it to unlock the private key (and remember it for the rest of the login session).

On macOS Sierra, the passphrases are not stored in standard login keychain and not synced with iCloud.

Managing Keys with multiple clients

If you have many client Macs you use to connect to the same hosts, you have two choices:

  • you can create a new key pair in each client and add the public key to the host’s authorized_keys file
  • you can copy the key pair from one client to another

The first option can grow unwieldly quickly when you are managing several Macs and several users.

The second option might expose your private key when you copy them from one client to another. If you send the private key unencrypted over the network it can be intercepted by a man-in-the-middle attack. If you copy it on a USB stick to transfer, you should remember to securely delete any copies, after the transfer.

Your organization will (should) have rules on how to setup and manage keys.


Given proper key management, client verification with keys is safer than with username and passwords. Because of this some setups only allow ssh connections an authorized key.

In this case you cannot use ssh-copy-id to transfer and add your public key to the host. You will have to provide the public key to an administrator to add it for you. The authorized_keys file will probably be in a different, central and locked-down location.

Automation with ssh and keys

You can write scripts that use ssh to communicate with another host. This can yield some powerful workflows. However, if the script uses ssh repeatedly, you do not want to have to enter the password over and over. If the script runs within a user session, ssh-agent can provide the passphrase when necessary. But when a script has to run in background on a schedule without user interaction, ssh-agent will not be available.

In this case you can create a key with an empty passphrase. This key will be unlocked automatically. Obviously, such a key would need to be protected very well, and its use should be audited closely. A key with an empty passphrase is the security equivalent of leaving the house key under the doormat. However, for some automated workflows, it may be the best solution.


  • previous post: ssh uses public/private keys to verify the host and encrypt the connection
  • ssh can also use public/private keys to verify the user connecting to the host
  • you generate a key with ssh-keygen
  • the public key is added to the remote host’s ~/.ssh/authorized_keys
  • the private key remains on the client and is locked with a passphrase
  • ssh-agent and Keychain integration on macOS can simplify access to the private key’s passphrase

Next post: Transferring Files with ssh

SSH Keys, Part 1: Host Verification

So far in this series of posts on ssh on macOS:

Please consider supporting Scripting OS X by buying one of my books!

In the earlier post, we talked about basic ssh setup and use.

We briefly mentioned ssh is securely encrypted. We encountered a cryptic prompt (pun intended) when connecting to the machine for the first time:

The authenticity of host ' (IP address)' can't be established.
ECDSA key fingerprint is SHA256:abcdefghhijklmonpqrstuvwxyz.
Are you sure you want to continue connecting (yes/no)? 

At that point we just wanted to connect and ignored this prompt. However, this is a crucial part of the ssh infrastructure and understanding it will help you use ssh securely and to its full potential.

Asymmetric Key Encryption

This is what ssh uses to encrypt traffic. It uses a two part encryption key to encode the data transmitted. The first part of the key is private and kept securely on the ssh host (the remote computer you want to log in to) and the second part is public and shared to the users who want to connect.

Data encrypted with one key can be decrypted with the other and vice versa. This way data can be kept secure between the client and the host.

When you connect for the first the remote host will send its public key to your computer. This is convenient, but opens the chance for malicious computer to impersonate the host you actually want to reach and set itself as a ‘man in the middle’, passing data back and forth and decrypting it in between.

OK, how does this work with SSH?

The key fingerprint the system displays on first connection is a ‘hash’ of the public key.

Using this fingerprint, you can verify that the DNS address or IP you are connecting to, is really the computer you want to connect to. To verify, you would need obtain the fingerprint from the host though some other means and compare.

On macOS the host keys are stored in /private/etc/ssh/ along with some other files required for ssh configuration. The key filenames have the format ssh_host_ABC_key where ABC is the key encryption type. On my Sierra Mac there are keys for dsa, ecdsa, rsa and ed25519. There two files for every type, the private key (no file extension) and the public key, with the .pub extension.

ECDSA (Elliptic Curve Digital Signature Algorithm – Wikipedia) is the default type of key ssh uses on macOS. The other keys are present for compatibility with other platforms and older versions of macOS/OS X. Usually the client and the server will negotiate which keys to use from the available options.

These keys are generated on every host, when the ssh server process starts for the first time. These keys are unique to this host.

When you look closely at the key files, you will see that the private keys can only be read by root, no other users. Private keys are like really important passwords and have to be kept safe. If someone can obtain both keys, they can impersonate this host.

The public keys are the part that are meant to be shared and can be read by any user.


You can generate the fingerprint from a public key with the ssh-keygen command:

$ ssh-keygen -l -f /private/etc/ssh/ 
256 SHA256:Izk7/TbLsH6aegfaGGs2hHQMAkDWkXR93gr1I/FKobo no comment (ECDSA)

ECDSA (Elliptic Curve Digital Signature Algorithm – Wikipedia) is the default type of key ssh uses on macOS. The other keys are present for compatibility with other platforms and older versions of macOS/OS X.

A security conscious admin would go to the remote machine in question (or ask another user/admin who has access to the server/machine) and generate the fingerprint. Then go back to the Mac from where he wants to remotely connect with ssh and compare the fingerprint shown on first connection:

$ ssh
The authenticity of host ' ([IP Address])' can't be established.
ECDSA key fingerprint is SHA256:Izk7/TbLsH6aegfaGGs2hHQMAkDWkXR93gr1I/FKobo.
Are you sure you want to continue connecting (yes/no)? 

Since the key and the fingerprint are unique, you can be sure you are talking to the correct remote machine and there is no other malicious computer impersonating the remote machine on the network.

Making a list, checking it twice…

Once you confirm the prompt to connect to a new host, its public key will be added to the file ~/.ssh/known_hosts. This file will contain one line per remote host. Each line will contain the hostname, the IP address, the key type and the public key data itself. You can view the file in a text editor, or you can use the ssh-keygen tool to search this file for a particular host:

$ ssh-keygen -F
# Host found: line 16,[IP Address] ecdsa-sha2-nistp256 AAAA1234...=

Once the key is stored in your known_hosts file, ssh can detect if the public key presented by a remote ssh host changes and warn you. The warning is not subtle:

$ ssh
The ECDSA host key for has changed,
and the key for the corresponding IP address [IP Address]
is unknown. This could either mean that
DNS SPOOFING is happening or the IP address for the host
and its host key have changed at the same time.
Someone could be eavesdropping on you right now (man-in-the-middle attack)!
It is also possible that a host key has just been changed.
The fingerprint for the ECDSA key sent by the remote host is
Please contact your system administrator.
Add correct host key in /Users/armin/.ssh/known_hosts to get rid of this message.
Offending ECDSA key in /Users/armin/.ssh/known_hosts:16
ECDSA host key for has changed and you have requested strict checking.
Host key verification failed.

In most cases this means that IP addresses have changed. In the worst case, someone is maliciously attempting to impersonate an ssh host in your network, probably in an attempt to gather login information.

As administrators, there is another common reason for this warning. When you re-install or re-image a client Mac, it will generate new ssh keys. So every time you re-image and re-connect to a test machine, your Mac will compare the old keys in known_hosts with a newly generated, different key on the freshly imaged Mac and bring up this warning.

You can remove the old keys from the ~/.ssh/known_hosts file directly in a text editor or use the ssh-keygen command to remove out of date keys:

$ ssh-keygen -R
# Host found: line 16
/Users/armin/.ssh/known_hosts updated.
Original contents retained as /Users/armin/.ssh/known_hosts.old

The hostname is used to identify hosts and keys in this file. If you connect to the client with different hostnames, DNS aliases, or you alternate between the ‘proper’ DNS name, partial DNS name mac, the Bonjour name (i.e. Mac.local), or the IP address, you may end up with multiple entries for the same host in known_hosts. Usually this is not a problem, but is useful to know when cleaning out stale keys.

If you have the public key for a host, you can pre-add it to your known_hosts file. The format you have to use is,[IP Address] ecdsa-sha2-nistp256 AAAA1234...=

Don’t copy these keys

If you use ‘Golden Master’ imaging on macOS (you really shouldn’t, for several reasons, this being one of them), you might inadvertently copy the host ssh keys from your Golden Master to all the other clients. While this does not break how ssh works, it will undermine some of its security. You can simply delete the existing keys on the Golden Master and new keys will be created on the clients when needed.

Remember there are other files in /private/etc/ssh/ that are required for ssh to work which you should not delete.

Locking down even more

You can increase or decrease the host key security with the StrictHostKeyChecking setting in the ~/.ssh/config file for a particular user or the/private/etc/ssh/ssh_config` file for all users on a system.

The default value is ask which means that ssh will ask what to do with an unknown key from a host. This is the behavior described above.

You can increase security by changing this setting to yes which means ssh will refuse to connect to a host which is not listed in ~/.ssh/known_hosts:

$ ssh
No ECDSA host key is known for and you have requested strict checking.
Host key verification failed.

With this setting you will have to manually obtain a host’s public key and add it to your known_hosts file.

You can also set StrictHostKeyChecking to no. With this setting, unknown hosts will automatically be added to the known_hosts file and changed public keys will automatically be replaced. This could be useful in a very dynamic, low-security environment (i.e. a classroom where Macs are re-imaged frequently). However, it does undermine the security of ssh and you will not notice any attempts to insert malicious hosts. In general you should not use this setting.


  • the ssh host (remote machine) has a set of encryption keys in /private/etc/ssh/
  • the public key is sent to the client. A fingerprint of the public key is presented with a prompt wether you should connect. You should use the public key fingerprint to verify the host’s authenticity on first connect.
  • generate the fingerprint from the public key on the host with ssh-keygen -l -f
  • the public keys of approved hosts are stored in ~/.ssh/known_hosts
  • use ssh-keygen -R host to remove old or unused keys
  • you can control this behavior with the StrictHostKeyChecking setting in the config file

When you verify the fingerprint of the host, you can be sure of its identity. The host however, cannot verify a client’s identity in this setup. This is why you still have to enter a password every time you log in with ssh remotely.

We will discuss how to improve on that in the next post : SSH Keys, Part 2: Client Verification

Quick Introduction to SSH for Mac Admins

Please consider supporting Scripting OS X by buying one of my books!

What is SSH?

SSH is short for ‘Secure SHell’. It is a protocol that allows you to open a shell (terminal session) on another remote computer over the network. On macOS you will usually initiate an SSH session with the Terminal application, though there are other third party applications for macOS and iOS that support the SSH protocol as well.

On macOS SSH access is sometimes called ‘Remote Login’.

As the name implies, connections over SSH are encrypted and secure. This may not seem unusual today, but it did replace many unsecured protocols such as telnet and rlogin.

SSH is supported on many platforms. You can use SSH to connect to Macs as well as any other computer which supports SSH and has it enabled. Of course, when you SSH to a different system, the environment you get through SSH might be very different (i.e. not bash, different commands, etc.), so be sure you understand the remote system.

Enable SSH access

By default SSH access is disabled on macOS. Before you can connect to a Mac with SSH you need to enable access. In the UI you can do this in System Preferences > Sharing, by enabling the ‘Remote Login’ option. Here you can also control whether all users on the Mac can get SSH access or just some of them.

The Sharing pane will also show the current hostname of this computer:

Computers on your local network can access your computer at:

You can also enable SSH access from the command line with

$ sudo systemsetup -setremotelogin on

and turn it off again with

$ sudo systemsetup -setremotelogin off

Turning SSH off with this command helpfully reminds you that if you are currently connected remotely, you will disconnect with this command and have to login locally to re-enable. To override this helpful notice, you can add the -f option:

$ sudo systemsetup -f -setremotelogin off

And if you want to know which hostname a Mac thinks it is, use the hostname command:

$ hostname

Note: If you want to control which users have access with a script, I have an example in this post. This post was written for Munki but should be fairly easy to adapt to other systems.

Connect with SSH

To connect to a machine from another Mac (the one you are working on) is simple enough. Open Terminal and enter:

$ ssh

or alternatively (different syntax, same result, which one you prefer is a matter of taste)

$ ssh -l username

If you do not give a username, ssh will use the short name you are currently logged in as. That is a useful shortcut if the username is the same on both systems.

When you connect to a remote machine for the first time you will get this prompt:

The authenticity of host ' (IP address)' can't be established.
ECDSA key fingerprint is SHA256:abcdefghhijklmonpqrstuvwxyz.
Are you sure you want to continue connecting (yes/no)? 

(Your actual fingerprint will look differently.) This prompt tells you that your ssh does not recognize this host and gives you a chance to not connect. For now, type yes to confirm. This will add the host to the list of known hosts, so the prompt will not return when you connect again. We will discuss keys and security in a later post.

Next ssh will prompt you for the password on the remote computer. Once you enter that you will get the prompt and have a secure shell to the remote computer. Any command you enter now, will be run on the remote computer.

You can connect with ssh when no user is logged in, i.e. the Mac is sitting at the login window. You can even connect with one user, while a different user is logged into the Mac with a UI session. In these cases, commands that interact with the UI, will fail, since the UI is either not running, or running as a different user. Most common examples are open and osascript (AppleScript).

$ open .
LSOpenURLsWithRole() failed with error -10810 for the file /Users/armin.

If you happen to be logged in on the same Mac with the same user, then the command will work, but open will open and display the Finder window on the remote Mac, not the Mac you are working on, which can be confusing.

Ending it

To end the remote session, just use the exit command.

$ exit
Connection to closed.

This will return you to the shell on your Mac where you executed the ssh command.

SSH Shortcut Files

If you frequently connect to certain hosts you may want to create an ssh short cut file, which you can then just double click or invoke with spotlight.

Execute Just One Command

Instead of opening a remote shell you can use ssh to just execute just a single command on the remote host:

$ ssh sw_vers
ProductName:    Mac OS X
ProductVersion: 10.12.5
BuildVersion:   16F73

The ssh command will prompt for the password and the print the output of the command from the remote machine. This can be very useful. You can combine multiple commands with ;

$ ssh hostname; sw_vers; system_profiler SPHardwareDataType

In some case the command you want to execute remotely can prompt for information, usually a password. You can add the -t option to make ssh use an interactive shell:

$ ssh -t sudo installer -pkg ~/Downloads/myinstaller.pkg -tgt / -verbose

This will prompt twice for the password. The first time to establish the remote connection and the second time for sudo on the remote machine.

Sending single commands with ssh can be useful for automating workflows in scripts. However, the requirement to keep entering passwords will be very detrimental to automation. It will also be annoying when you frequently connect to specific remote Macs.

We will discuss how the key security works and how it can replace passwords in the following posts.

Check all AutoPkg Recipes

AutoPkg needs to load all recipes in the search paths on every run so it can locate recipes and their parents (which may be spread over different repositories).

Because of this, AutoPkg may fail if a single recipe has malformed property list syntax. To locate the one broken property list among many, you can use the following command:

$ find ~/Library/AutoPkg -name "*.recipe" -exec plutil -lint {} \; | grep -v "OK$"

This uses find to find all files with the .recipe file extension in the AutoPkg folder and executes plutil -lint filename, then it uses grep to show only lines not (-v) ending with ‘OK’.

Post WWDC Summary

Earlier this year I wrote a post on whether packaging is dead. Since I wrote a book on Packaging and have also invested much of my career in macOS I do have quite some interest in the topic.

(Please buy the book! If you have bought and read it, please leave a review!)

After that post I made myself a reminder to re-visit the topic post WWDC. I was very much expecting to be proven wrong or hopelessly optimistic. This reminder has been bugging me for a while. I have had a hard time to consolidate my thoughts into writing.

It’s not that this year’s WWDC was boring. Quite the opposite. The new iPads Pro look wonderful and I want one. Apple also announced great new iMacs and MacBooks and a space-grey iMac Pro, demonstrating they still care about the Mac line. (The Mac mini, however, got no love this time around. I do hope the line gets at least a speed bump and we don’t have to wait for a the new Mac Pro to get a decent option for screenless Macs. I’ve given up on servers…) And finally, both iOS 11 and macOS High Sierra (10.13) look like solid updates with lots of new features for users and developers. This was a great WWDC!

Mac admins were concerned that this update would lock down macOS in a similar fashion to iOS. The worst case scenarios painted a picture where not even admin users would be able to get root privileges and you couldn’t install third party daemons and agents any more, fundamentally breaking the way all management systems work. Admins would have to re-work their workflows to work through MDMs, which are not yet capable to bear this burden. The new Apple File System APFS would break NetBoot and all the tools admins use to image Macs.

What happened was… well… nothing much really.

Mac-narök has been postponed.

(Excellent talk by Micheal Lynn at MacDevOps YVR, just a few minutes before the WWDC Keynote. Go watch it.)

There will be changes in High Sierra that affect admins. APFS on macOS is definitely going to happen. In the current (first) beta there is an option to disable the filesystem conversion during upgrade, but it is unknown wether that option will still exist in the release. You can now add iOS devices to DEP even if they were not registered at purchase. You can control a firmware password on Macs with profiles. There are some (minor) changes to files and folders protected by SIP.

I don’t believe or want to suggest the posts above and many other people who predicted the end of Mac Administration as we know it were hysterical or unnecessarily panicked. When they were written there were strong indications and hints that Apple was planning a lockdown of some form soon. MDM only Mac administration might still happen in a future update. However, we seem to have gotten an reprieve, which is good.

Why did the lockdown not happen now? Excellent question, which I do not know the answer to. There was a big outcry from the Mac admin community and many used their official channels (Apple reps and support, Radar, Feedback) to tell Apple what a huge imposition such a quick and drastic change would be. Also many third-party application developers are reluctant to (or cannot) move to the Mac App Store, which would be a requirement in an MDM only world.

For now it seems that common admin tools will run on High Sierra and APFS with just some minor adjustments. This includes packages! Packaging is not dead! Long live Packaging! …and all the other tools!

(On the other hand, some things may still break or be removed during the beta phase.)

Does that mean we should just happily keep doing what we are doing? No. Even if Apple does not yet enforce ‘MDM-only’ they are clearly moving towards ‘more MDM.’ We still have to re-evaluate every setting and workflow with MDM in mind. There are some great solutions already that can combine MDM with e.g. Munki, Chef or Puppet.

Even though imaging, whether you choose the “thick” or “thin” approach, will probably still work in High Sierra, you should be thinking about alternative strategies. DEP plus application installs and updates are more flexible and powerful than full disk imaging.

There are certain setups, such as classrooms and training centers, which require frequent re-imaging with short turnaround times. Ironically, the tech that was predicted to kill imaging might provide a solution. APFS disk snapshots could provide a solution for fast system restores. The tools for this do not seem to be fully in place yet, but the time to test and file bugs is now.

The MDM ‘InstallApplication’ command, which installs the agent software, such as the Munki or Jamf client, should be supported by management systems. This would allow clients to be connected to the management system without user interference and the client software to add to the limited functionality of MDM with tools that admins already have solutions and expertise for.

So the post WWDC summary: the ‘End of Things as We Know Them’ has been postponed. Imaging will still work, but you want to start examining and testing alternatives. Packages and scripts remain relevant, but there are interesting new means of distributing them.

It is already apparent the next WWDC will have more exciting news ready for Mac and iOS Admins. Until then we will be busy learning the new features and tools in High Sierra and iOS 11 and laying the groundwork to the future.

Tab Completion for autopkg

Tony Williams aka ‘honestpuck’ has built a script to enable tab-completion for autopkg in bash.

This means that you can type

$ autopkg s⇥

(where ⇥ is the tab key) and it will autocomplete to

$ autopkg search 

This will also work for recipe names:

$ autopkg run BBEdit⇥⇥  BBEdit.jss       BBEdit.pkg       
BBEdit.install   BBEdit.munki     

This is really useful. Auto-completion not only saves on typing, but helps to avoid errors.

Installing autocompletion in your profile

Tony has provided instructions on how to install the script with brew. However, it not hard to install this manually in your .bash_profile or .bashrc. First, clone the github repository on to your system (I keep all projects like this in an un-creatively named ‘Projects’ folder):

$ cd ~/Projects
$ git clone

This will download the project to autopkg_create. The file we need is the autopkg file inside that folder.

Then add the following lines to your .bash_profile or .bashrc:

if [[ -r "$HOME/Projects/autopkg_complete/autopkg" ]]; then
    source "$HOME/Projects/autopkg_complete/autopkg"

You will need to adjust the path if you are using a different location. Basically these lines say: if this file exists and is readable, then read and interpret it as bash source. Since you need to define functions in the context of the shell, you need to `source` the file, rather execute it as script. (When you run the the file as a script, the functions will be defined in the context of the script, and then ‘forgotten’ when the script ends.)

Save your new profile and open a new Terminal window or type

$ source ~/.bash_profile

to update an existing shell.

Thanks again to Tony Williams, this is very useful!

Where PATHs come from

In an earlier post we talked about how to append to the PATH variable, so you can add your own directories to bash’s search path.

In macOS the default PATH on a ‘clean’ installation is:

$ echo $PATH

However, if you have installed some tools (such as the macOS, Xquartz or Munki) you will see those in the PATH as well:

$ echo $PATH

Where does this pre-set PATH come from?

Since the PATH is pre-set on a clean new account without a .bashrc or .bash_profile, we have to look in a central location which applies to all users. In another earlier post, we saw that /etc/profile is run for every user shell, even before a .bash_profile is executed. When you look into this file, you see that the very first set of commands look like this:

if [ -x /usr/libexec/path_helper ]; then
    eval `/usr/libexec/path_helper -s`

This looks very promising. The path_helper tool has a man page. This tool does a few things to assemble the PATH. First it reads the file /etc/paths which on macOS looks like this:


So this is where the default ‘clean’ macOS PATH comes from. Then path_helper will read every file from /etc/paths.d and append each line of each file in that directory to the PATH as well. This is where optional and third party applications and tools, like Xquartz or Munki, can install their own additions to the PATH for all users.

(Files in this folder will be read in alpha-numerical order of the filename. Some tools, like Xquartz, attempt to influence the order by preprending a number, e.g. 40-XQuartz.)

Finally, if path_helper runs in an environment where PATH is already set, it will append that PATH value to what it built from the files and then remove duplicates.

path_helper does not change the environment variable directly, but it generates the commands necessary to set the PATH correctly. It will generate the right commands wether it is called from a bourne type shell (on macOS: sh, bash, ksh and zsh) or a csh type shell (on macOS: csh and tcsh). You can see the output of the two styles by running path_helper with the -s or -c options:

$ /usr/libexec/path_helper -s
PATH="/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin"; export PATH;
$ /usr/libexec/path_helper -c
setenv PATH "/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin";

(You have to type the full path to path_helper because, ironically, but intentionally, /usr/libexec is not in the standard PATH.)

To actually execute the commands generated by path_helper you can use the eval command, like the /etc/profile does:

$ eval $(/usr/libexec/path_helper)

Don’t touch my Profile!

Some command line tool installers understandably feel the need to add their tools to the default PATH. Since there is no unified approach among different flavors of UNIX and Linux on how to do this, you will find several different approaches. Some tools will edit /etc/profile and others will look for the various profile files in a user’s home directory and edit those. Usually the installation process will append a line that appends their tools directory to the PATH.

One example for this is the Python 3 installer. It contains a compnent package that will attempt to determine which profile file you are using and appends a line to append to the PATH.

However, this is not only highly intrusive but also quite fragile. Changes to /etc/profile might be overwritten by a future macOS update. Changes to a user’s profile file, might be overwritten by the user. Also the installer will only append their setting to the current user, not other users that may be present or created in the system.

Sample paths.d installer package

On the other hand, dropping a file into /etc/paths.d with a package installer will affect all users on a system. The file in paths.d can be updated for future updates if necessary and is also easily identified and removed by an admin. It will work wether it is installed to the startup volume or another volume. It can be pushed with management tools.

Building an installer for a file in /etc/paths.d is very simple:

$ mkdir CustomToolPathInstaller
$ cd CustomToolPathInstaller
$ mkdir payload
$ echo "/usr/local/customtool" >> payload/customtool
$ pkgbuild --root payload --install-location /private/etc/paths.d --version 1.0  --identifier com.example.customtool.path CustomToolPath.pkg
pkgbuild: Inferring bundle components from contents of payload
pkgbuild: Wrote package to CustomToolPath.pkg

Only five commands, three of which create the folder structure. You can find this sample project (which is slightly more elaborate) on my GitHub.

If you want to learn more about building installer packages for macOS, please read my book “Packaging for Apple Administrators”.

What about MANPATH?

This is usually not used on macOS since the the default settings for the man tool are quite flexibel. (Look at the man page for man and the file /etc/man.conf for details.) However, if a MANPATH environment variable is set when path_helper runs, it will also assemble the command to set the MANPATH built in a similar way to the PATH from the files /etc/manpaths and the directory /etc/manpaths.d.

Usually the MANPATH is not set on macOS so you will not see this. But if you want to manage your MANPATH and want to leverage path_helper all you have to do is set the MANPATH.

$ export MANPATH=/usr/share/man
$ /usr/libexec/path_helper
PATH="/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin"; export PATH;
MANPATH="/usr/share/man:/usr/local/share/man"; export MANPATH;

Re-order the PATH

We have seen path_helper is extremely useful. There is one caveat, however. path_helper may reorder your PATH. Imagine you are pre-pending ~/bin to your PATH because you want to override some standard tools with your own. (Dangerous, but let’s assume you know what you are doing.) Then some process launches a subshell which can call path_helper again. path_helper will ‘find’ your additions to the defined PATH, but it will append to the list of default paths from /etc/paths and /etc/paths.d, changing your order and thus which tools will be used.

$ export PATH=~/bin:$PATH
$ echo $PATH
$ /usr/libexec/path_helper 
PATH="/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin:/Users/armin/bin"; export PATH;

You can see behavior like this when you use Xterm (The X11 based terminal in Xquartz) which does not execute .bash_profile but still picks up the PATHenvironment variable from somewhere…

# in Xquartz Terminal:
bash-3.2$ echo $PATH

A better way to override built-in commands which is not affected by path_helper would be to use bash aliases or functions in your profile.

Configuring bash with aliases and functions

In the previous posts we talked about which files you could use to customize your bash environment.

There are (mainly) three customizations you can perform:

This post will look at some useful aliases and functions.

bash aliases

Note: bash aliases are something completely different from Finder aliases. The closest shell equivalent for Finder alias files are symbolic links.

bash aliases are basically text substitutions. For example a common alias is to define:

alias ll="ls -l"

You can type this alias definition directly into bash but then the definition will only exist for that particular shell. When you open a new terminal window it will know nothing of this alias. If you want an alias to exist in all your shells (and usually you do), you need to add the above line to your .bash_profile or .bashrc. (Learn about the difference here.)

Whenever you modify the .bash_profile or .bashrc it will not automatically be loaded into any shell that is already open, you either have to close the Terminal window and open a new one or write

$ source ~/.bash_profile

(or ~/.bashrc) to load the new settings.

Once you have set the alias, anytime you type ll at the start of a command, bash will replace it with ls -l before executing. Since subsequent arguments are left alone they will just be picked up by the substituted ls command, so if you type ll -a bash will substitute that to ls -l -a and it will work just your would expect.

You can make the alias more complex if you want:

alias lll="ls -alhTOe@"

If you always want to use the long format of ls, you can alias the ls command itself:

alias ls="ls -l"

Then, when ever you type ls it will be replaced with ls -l. Note the lack of spaces around the ‘=’, as usual when assigning values in bash.

Uncovering the alias

You can unset or delete an alias with the unalias command:

$ unalias ls

This will return to the default ls command (for this bash instance only).

Since alias substitution only happens at the beginning of the command, you can also bypass it once by starting the command line with a different character:

$ "ls" -S
$ \ls -S

Either way of typing this will use the original ls command, avoiding substitution, just this once.

If you are unsure if a command has an alias or want to know what gets substituted, then you can have the alias definition printed with

$ alias ls
alias ls='ls -l'
$ alias lll
alias lll='ls -alhTOe@'

You can also list all defined aliases by running alias without any arguments:

$ alias
alias ll='ls -l'
alias lll='ls -alhTOe@'
alias ls='ls -l'

Some Alias examples

Some users like to alias the potentially dangerous commands rm, mv and cp with the -i option, which forces the user to confirm when a file is going to be deleted or overwritten:

alias rm="rm -i"
alias mv="mv -i"
alias cp="cp -i"

Then if you do something that could destroy an existing file you will be prompted to confirm:

$ rm my_important_file 
remove my_important_file? 

You can still use \rm to bypass the alias, if you believe you know what you are doing and want to avoid being prompted several times.

You can add short cuts to cd to parent folders:

alias ..="cd .."
alias ...="cd ../.."
alias cd..="cd .."

Since alias substitution only takes place at the beginning of the command, you can still use .. normally as an argument.

Note that the last alias cd.. is a substitution of a common typo. Since the output of the alias is not checked for further alias substitutions you cannot use alias cd..=.. to define it using the previous alias. Each alias must stand for itself.

You can also use aliases to make complex or hard to memorize commands easier to remember:

alias badge="tput bel"

The first command will play a system alert and set a ‘badge’ on the terminal application if it is in the background. This is useful to get notified of long running commands:

$ hdiutil convert image.sparseimage -format UDZO image.dmg; badge;

The substitution can be long and multiple commands can be separated by ;

alias dockspace="defaults write persistent-apps -array-add '{\"tile-type\"=\"spacer-tile\";}'; killall Dock;"

The dockspace alias uses this hack to add a spacer item to the dock.

We have seen earlier with the ll alias that any text or arguments after the alias will just be added to the substituted command. We can use this to our advantage:

alias reveal='open -R'
alias xcode='open -a Xcode'
alias pacifist='open -a Pacifist'

All of these commands will require a file path as an ‘argument’ to work properly.

Functions: beyond the alias

While bash aliases are useful and easy to define, they have limitations. Aliases will only be replaced at the beginning of the command prompt. Also additional arguments will be appended after the replacement. If you need to insert additional arguments somewhere in the replacement, you cannot achieve that with aliases.

For example, to display the man page for ls in the Preview application you need the following commands:

$ man -t ls | open -f -a "Preview"

If you wanted to alias this you would have to insert the arguments after man -t and before the pipe. Since aliases, only substitute the beginning of the command, this will not work. we need to define a bash function:

function preman() {
    man -t $@ | open -f -a "Preview"

Sometimes you will see bash functions defined without the function keyword, but I prefer to use it since it makes the code more readable.

The $@ will be replaced with all the arguments passed into the function, so when use the function

$ preman ls

the argument ls will be substituted where the $@ is.

You can also use the positional argument variables $1, $2, etc:

function recipe-open() {
    open "$(autopkg info '$1' | grep 'Recipe file path' | cut -c 22-)"

Within functions the full power of bash scripting is at your service. You can even call other scripting languages.

# prints the path of the front Finder window. Desktop if no window open
function pwdf () {
    osascript <<EOS
        tell application "Finder"
            if (count of Finder windows) is 0 then
                set dir to (desktop as alias)
                set dir to ((target of Finder window 1) as alias)
            end if
            return POSIX path of dir
        end tell

# changes directory to frontmost 
alias cdf='pwdf; cd "$(pwdf)"'

The last alias cdf has to be defined as an alias. Since a function or script could not change the directory for the current shell, you have to use alias substitution to get the cd.

In general functions (and scripts) are more powerful and versatile than aliases, but aliases provide and easy and comparatively safe way to customize the your shell environment.

Putting it all together

Here is a sample .bash_profile with many of the examples given here and in the previous article. You can use this as a starting point for your own bash configuration.

On bash Environment Variables

This is vacation time with a holiday in the Netherlands. Posting will be more sporadic than usual.

In the previous post we talked about which files you could use to customize your bash environment.

There are (mainly) three customizations you can perform:

This post will look at some useful environment variables.

Choose your own PATH

The PATH environment variable contains a list of directories that bash will search through for commands. You can show your current path by echoing the PATH variable:

$ echo $PATH

This is the default PATH for a ‘clean’ macOS. Depending on what you have installed, there may be more elements in your PATH. The individual directories are separated by a colon :. When you enter a command, like for example systemsetup at the prompt, bash will search through the directories in PATH in the order until it finds a match.

Note: if there is ever any question on which command is actually executed, you can use the which command to get the effective path to a given command from the shell:

$ which python
$ which systemsetup
$ which bash

You might have installed other (newer versions) of certain tools (such as python or bash itself). Depending on how you install them they may be installed in a way that overrides the tool that is part of macOS. This may lead to problems when scripts assume the system version of a tool.

You can add your own directories by either appending (safe) or prepending (risky) your own directories to the PATH variable:

export PATH

This will add the ~/bin directory (which does not exist by default, you have to create and fill your own) to your PATH. The export command tells bash it should export this variable to child processes.

You can add more than one directory to the PATH, either at once, or one per line:

export PATH=$PATH:~/bin:~/Projects/scripts


export PATH

Since bash stops looking when it finds a match, the order of the directories in the PATH is important. When you place your directories before the default directories you can override some of the system commands. This can be powerful, but also very dangerous. There are safer methods for overriding system commands. You should add your directory to the end of the PATH to be safe.

Note: the PATH (and other environment variables) are exported to child-shells and scripts executed from the shell. However, not every tool executes directly from a shell, or from a shell with your configuration file. So you should never assume PATH (or any other environment variable) is set to anything specific (or even set) when you write a script. You can either set the PATH at the beginning of a script, or use absolute paths for commands in scripts such as /usr/bin/defaults or /usr/bin/pkgbuild. You can determine the path to a command with the which command.

Prompt Me!

You can also configure the command prompt, i.e. the text that bash displays before you enter a command. By default the macOS prompt looks like this:

hostname:currentDir user$ 

The configuration for the prompt is stored in the PS1 environment variable. You can see the default value by echoing it:

$ echo $PS1
\h:\W \u\$

The letters with the backslash \ in the prompt variable are placeholders that will be replaced with a specific value when the prompt is printed out.

  • \h will be replaced with the hostname of the computer up to the first . (\H would be the complete hostname)
  • \W will be the basename of the current working directory
  • \u is the current user
  • \$ will be $ unless the current user is root or effectively elevated to root privileges, in which case it will be #. This is very useful as a warning when working with elevated privileges.

Since I am the only user on my Mac, and I also know which Mac I am sitting at, I have changed my prompt to something simpler:

export PS1="\W \$ "

Note the space after the \$. If you forget that the cursor will stick right on the $.

(When I log in to a different Mac with ssh the different prompt also reminds me that this shell is on a different Mac.)

You can find a list of escape sequences here.

You can also add color to the prompt data, but the escape sequences will look very messy very quickly. To have the directory name in gray, you use:

export PS1="\[\e[1;30m\]\W\[\e[m\] \\$ "

List in Color

You can also tell bash to color for some other commands, such as ls. To do that you just have to set the CLICOLOR variable:

export CLICOLOR=1

Change the Editor

Some terminal commands will open a text editor directly. For example git may open a text editor so you can enter a commit message. By default the system will open vi but you can change the editor that will be used. There are even some graphical editors that can be used this way. The environment variable for this behavior is EDITOR.

vi or vim:

export EDITOR=vi
export EDITOR=vim


export  EDITOR=emacs

nano or pico:

export EDITOR=nano
export EDITOR=pico


export EDITOR="open -f"


export EDITOR="bbedit -w --resume"


export EDITOR="mate -w"


export EDITOR="subl -n -w"


export EDITOR="atom --wait"

Note: the -w or --wait argument tells the UI application that there is another process waiting for their output.


Putting all of this together, you can add these lines to your `.bash_profile` or `.bashrc`:

Up Next?

In the next post we will look at bash aliases and functions.

About bash_profile and bashrc on macOS

Note: bash_profile is completely different from configuration profiles. Learn more about Configuration Profiles in my book: ‘Property Lists, Preferences and Profiles for Apple Administrators’

You can learn more about using Terminal and the shell on macOS in my my book: “macOS Terminal and Shell” — Thank you!

In this spontaneous series on the macOS Terminal I have often mentioned adding something as an alias or function to your bash_profile or bashrc. Obviously, you may wonder: how do I do that? And which file should I use?


When you work with the command line and the bash shell frequently, you will want to customize the environment. This can mean changing environment variables, such as where the shell looks for commands or how the prompt looks, or adding customized commands.

For example, macOS sets the PATH environment variable to /usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin by default. This is a list of directories (separated by a colon ‘:’) that the system searches through in order for commands. I like to add a folder in my home directory ~/bin to that list, so that I can execute certain tools without needing to type out the full path. (e.g. munkipkg, quickpkg and ssh-installer).

In bash you append to existing PATH do this with:

export PATH="$PATH:~/bin"

You could type this command every time you open a new Terminal window (i.e. shell), or you can configure your shell to do this automatically.

Depending on which shell you use and how you start the shell, then certain script files will be executed which allow you to set up these customizations.

This article will talk about customizing bash on macOS. Other shells and other operating systems may have other files or rules.

So, which file?

Thanks to the rich and long history of bash the answer to which file you should put your configuration in, is surprisingly confusing.

There are (mainly) two user level files which bash may run when a bash shell starts. ~/.bash_profile or ~/.bashrc.

Both these files are on the first level of your home directory ~/. Since the file names start with a . Finder and normal ls will not show them. You need to use ls -a to see if they are present. Read more about invisible and hidden files here.

The usual convention is that .bash_profile will be executed at login shells, i.e. interactive shells where you login with your user name and password at the beginning. When you ssh into a remote host, it will ask you for user name and password (or some other authentication) to log in, so it is a login shell.

When you open a terminal application, it does not ask for login. You will just get a command prompt. In other versions of Unix or Linux, this will not run the .bash_profile but a different file .bashrc. The underlying idea is that the .bash_profile should be run only once when you login, and the .bashrc for every new interactive shell.

However, on macOS, does not follow this convention. When opens a new window, it will run .bash_profile. Not, as users familiar with other Unix systems would expect, .bashrc.

Note: The Xterm application installed as part of Xquartz runs .bashrc when a new window opens, not .bash_profile. Other third-party terminal applications on macOS may follow the precedent set by or not.

This is all very confusing.

There are two main approaches:

  • When you are living mostly or exclusively on macOS and the, you can create a .bash_profile, ignore all the special cases and be happy.
  • If you want to have an approach that is more resilient to other terminal applications and might work (at least partly) across Unix/Linux platforms, put your configuration code in .bashrc and source .bashrc from .bash_profile with the following code in .bash_profile:
if [ -r ~/.bashrc ]; then
   source ~/.bashrc

The if [ -r ... ] tests wether a file exists and is readable and the source command reads and evaluates a file in place. Sometimes you see

[ -r ~/.bashrc ] && . ~/.bashrc

(mind the spaces) Which is a shorter way to do the same thing.

Since either file can drastically change your environment, you want to restrict access to just you:

$ chmod 700 ~/.bash_profile
$ chmod 700 ~/.bashrc

That was confusing. Is that all?

No. There are more files which may be executed when a shell is created.

When bash cannot find .bash_profile it will look for .bash_login and if that does not exist either .profile. If .bash_profile is present the succeeding files will be ignored. (though you can source them in your .bash_profile)

There is also a file /etc/profile that is run for interactive login shells (and This provides a central location to configure the shells for all users on a system. On macOS /etc/profilesets the default PATH with the path_helper tool and then sources /etc/bashrc which (you guessed) would be the central file for all users that is executed for non-login interactive shells. For macOS /etc/bashrc sets the default prompt and then itself sources /etc/bashrc_Apple_Terminal which sets up the session persistence across logins.

So in macOS, before you even see a prompt, these scripts will be run:

  • /etc/profile
    • /etc/bashrc
      • /etc/bashrc_Apple_Terminal
  • if it exists: ~/.bash_profile
    • when ~/.bash_profile does not exists, ~/.bash_login
    • when neither ~/.bash_profile nor ~/.bash_login exist, ~/.profile
  • ~/bash_profile can optionally source ~/.bashrc

There is also a file ~/.inputrc, where you can setup certain command line input options. One common example for this is to enable case-insensitive tab-completion. You can find a list of more options here.

Finally, there is ~/.bash_logout which is run when a shell exits or closes.

Ok, so I have the file, now what?

Whichever file you choose, (I went with option one and have everything in .bash_profile) now you want to put stuff in it.

Technically this is a script, so you can do anything you can code in bash. However, usually the contents of a .bash_profile or .bashrc fall into one of three categories:

I will show some examples for each in the next post!