Weekly News Summary for Admins — 2018-02-16

There was some news this week about Apple refocusing the next major OS updates on bug fixing. Will this be (High) Sierra’s Snow Leopard?

Getting ready for my MacAD.UK presentation next week. Looking forward to meeting some of you there!

If you would rather get the weekly newsletter by email, you can subscribe to the Scripting OS X Weekly Newsletter here!! (Same content, delivered to your Inbox once a week.)

#! On Scripting OS X

📰News and Opinion

🐞Bugs and Security

🔨Support and Tutorials

🤖Scripting and Automation

🍏Apple Support

♻️Updates and Releases

🎧To Listen

📚Support

I do not have any ads on my webpage or this newsletter. However, if you want to support me and this website, then please consider buying one (or both) of my books. (Imagine it’s like a subscription fee, but you also get one or two useful books on top!)

If you have already bought and read the books, please leave a review on the iBooks Store. Reviews are important to help new potential readers make the purchase decision. Thank you (again)!

Setting the PATH in Scripts

A discussion that comes up frequently on MacAdmin Slack and other admin discussions is:

Should commands in scripts have their full path hardcoded or not?

Or phrased slightly differently, should you use /bin/echo or just echo in your admin scripts?

I talked about this briefly in my MacSysAdmin session: Scripting Bash

Why can’t I just use the command?

When you enter a command without a path, e.g. echo, the shell will use the PATH environment variable to look for the command. PATH is a colon separated list of directories:

$ echo $PATH
/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin

The shell will look through these directories in order for the given command.

You can read more detail about the PATH and environment variables in these posts:

PATH is Unreliable

The example PATH above is the default on macOS on a clean installation. Yours will probably look different – mine certainly does:

$ echo $PATH
/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin:/Applications/VMware Fusion.app/Contents/Public:/usr/local/munki:/Users/armin/bin

Third party applications and tools can and will modify your PATH. You yourself might want to change your PATH in your shell profile.

But on top of that, the PATH will be different in different contexts. For example, open the Script Editor application, make a new script document, enter do shell script "echo $PATH" and run the script by hitting the run/play button.

The small AppleScript we just built runs the shell command echo from the AppleScript context. The result is

/usr/bin:/bin:/usr/sbin:/sbin

Note how the PATH in this context is different from the default macOS PATH in Terminal and also different from your PATH.

I also built a simple payload-free installer package which only runs the echo "Installer PATH: $PATH" command. You will have to search through /var/log/install.log to get the output:

installd[nnn]: ./postinstall: Installer PATH: /bin:/sbin:/usr/bin:/usr/sbin:/usr/libexec

Which is yet another, different PATH.

Solutions to the PATH confusion

The PATH may be different in different contexts that your script may run in.

That means we cannot be consistently certain if a command will be or which command will be found in a script in different contexts. This is, obviously, bad.

Mac system adminstrative scripts which can run in unusual contexts, such as the Login Window, NetInstall, the Recovery system or over Target Disk Mode and usually run with root privileges. You really want to avoid any uncertainty.

There are two solutions to make your scripts reliable in these varying contexts:

  1. hardcode the full path to every command
  2. set the PATH in the script

Both are valid and have upsides and downsides. They are not exclusive and can be both used in the same script.

Going Full PATH

You can avoid the unreliability of the PATH by not using it. You will have to give the full path to every command in your script. So instead of

#!/bin/bash

echo "hello world"

you have to use:

#!/bin/bash

/bin/echo "hello world"

When you do not know the path to a command you can use the which command to get it:

$ which defaults
/usr/bin/defaults

Note: the which command evaluates the path to a command in the current shell environment, which as we have seen before, is probably different from the one the script will run in. As long as the resulting PATH starts with one of the standard directories (/usr/bin, /bin, /usr/sbin, or /sbin) you should be fine. But if a different PATH is returned you want to verify that the command is actually installed in all contexts the script will run in.

Using full paths for the commands works for MacAdmin scripts because Mac administrative scripts will all run on some version of macOS (or OS X or Mac OS X) which are very consistent in regard to where the commands are stored. When you write scripts that are supposed to run on widely different falvors of Unix or Linux, then the location of certain commands becomes less reliable.

Also note that some elements of shell scripting are commands, even if they don’t look like it. For example the square bracket [ is actually a command:

$ which [
/bin/[

so if you are using full command paths, you should use if /bin/[ … instead of if [ ….

(On the other hand, you should be using double brackets [[ instead of single brackets for tests, since they are safer and more flexible, but that is a topic for another post.)

Choosing your own PATH

The downside of hardcoding all the command paths is that you will have to memorize or look up many command paths. Also, the extra paths before the command make the script less legible, especially with chained (piped) commands.

If you want to save effort on typing and maintenance, you can set the PATH explicitly in your script. Since you cannot rely on the PATH having a useful value or even being set in all contexts, you should set the entire PATH.

This should be the first line after the shebang in a script:

#!/bin/bash
export PATH=/usr/bin:/bin:/usr/sbin:/sbin

echo "hello world"

Note: any environment variable you set in a script is only valid in the context of that script and any sub-shells or processes this script calls. So it will not affect the PATH in other contexts.

This has the added benefit of providing a consistent and well known PATH to child scripts in case they don’t set it themselves.

The downside of this is that even with a known PATH you cannot be entirely sure which tool will be called by the script. If something installed a modified copy of echo in /usr/bin it would be called instead of the expected /bin/bash.

However, on macOS the four standard locations (/usr/bin, /bin, /usr/sbin, /sbin, as well as the less standard /usr/libexec) are protected by System Integrity Protection (SIP) so we can assume those are ‘safe’ and locked down.

/usr/local/bin is a special case

But notice that I do not include /usr/local/bin when I set the PATH for my scripts, even though it is part of the default macOS PATH. The PATH seen in the installer context does not include /usr/local/bin, either.

/usr/local/bin is a standard location where third party solutions can install their commands. It is convenient to have this directory in your interactive PATH under the assumption that when you install a tool, you want to use it easily.

However, this could create conflicts and inconsistent results for administrative scripts. For example, when you install bash version 4, it will commonly be installed as /usr/local/bin/bash, which (with the standard PATH) overrides the default /bin/bash version 3.

Since you chose to install bash v4, it is a good assumption that you would want the newer version over the older one, so this is a good setting for the interactive shell.

But this might break or change the behavior of administrative scripts, so it is safe practice to not incluse /usr/local/bin in the PATH for admin scripts.

Other Tools

When you use commands from other directories (like /usr/libexec/PlistBuddy, or third party tools like the Munki or Jamf tools) then it is your choice whether you want to use full path for these commands or (when you use the commands frequently in a script) add their directory to the PATH in your script:

E.g. for Munki

export PATH=/usr/bin:/bin:/usr/sbin:/sbin:/usr/local/munki

or Jamf

export PATH=/usr/bin:/bin:/usr/sbin:/sbin:/usr/local/jamf/bin

Since the third party folders are not protected by SIP, it is safer to append them to the PATH, so they cannot override built-in commands.

Commands in Variables

Another solution that is frequently used for single commands with long paths is to put the entire path to the command in a variable. This keeps the script more readable.

For example:

#!/bin/bash

# use kickstart to enable full Remote Desktop access
# for more info, see: http://support.apple.com/kb/HT2370

kick="/System/Library/CoreServices/RemoteManagement/ARDAgent.app/Contents/Resources/kickstart"

#enable ARD access
$kick -configure -access -on -users remoteadmin,localadmin -privs -all
$kick -configure -allowAccessFor -specifiedUsers
$kick -activate

Note: that you need to quote the variable when the path to the command contains spaces or other special characters.

Summary

As a system administrator it is important to understand the many contexts and environments that a script might be run in.

Whether you choose to write all command paths or explictly set the PATH in the script is a matter of coding standards or personal preference.

You can even mix and match, i.e. set the PATH to the ‘default four’ and use command paths for other commands.

My personal preference is the solution where I have to memorize and type less, so I set the PATH in the script.

Either way you have to be aware of what you are doing and why you are doing it.

All the Articles

While organizing links for my upcoming MacAD.UK presentation, I noticed that there are quite a few series of articles I have written over the past year or so. Some are weren’t quite intended as series but turned into loose sequels. Some were intended as series from the start, but usually turned out longer than intended.

Anyway, while I was organizing links anyway I also created a new page on this site that organizes the series:

Article Series on Scripting OS X

I intend to keep it updated as new articles and series are added.

Enjoy!

Weekly News Summary for Admins — 2018-02-09

No really big news this week. But many Mac Admins took time to write quite a few interestings articles and how tos. Thank you!

If you would rather get the weekly newsletter by email, you can subscribe to the Scripting OS X Weekly Newsletter here!! (Same content, delivered to your Inbox once a week.)

#! On Scripting OS X

📰News and Opinion

🔨Support and HowTos

🍏Apple Support

♻️Updates and Releases

🎧To Listen

📚Support

I do not have any ads on my webpage or this newsletter. However, if you want to support me and this website, then please consider buying one (or both) of my books. (Imagine it’s like a subscription fee, but you also get one or two useful books on top!)

If you have already bought and read the books, please leave a review on the iBooks Store. Reviews are important to help new potential readers make the purchase decision. Thank you (again)!

Show the Installer Log During High Sierra Installation

In previous versions of macOS you could show the installer log during system installation with the keyboard shortcut ⌘L. Since macOS Sierra, much of the system installation process happens on a black screen with a white Apple logo and a progress bar.

On that screen, you can use the keyboard shortcut ⇧⌃⌥⌘W, and it will switch back to traditional UI with the grey window. There you can then use ⌘L to show the log window again.

(Double mouse pointers are because I was running the installer in a VM to record the screen.)

Weekly News Summary for Admins — 2018-02-02

This week was a bit quieter and given the news from the last few weeks, that is probably a good thing. Quite a few very interesting and useful articles, though.

Ed Marczak has put together a great site with information for the ‘new’ Mac Admin: Mac Admin Info

If you would rather get the weekly newsletter by email, you can subscribe to the Scripting OS X Weekly Newsletter here!! (Same content, delivered to your Inbox once a week.)

#! On Scripting OS X

📰News and Opinion

🔨Support and HowTos

🍏Apple Support

♻️Updates and Releases

📚Support

I do not have any ads on my webpage or this newsletter. However, if you want to support me and this website, then please consider buying one (or both) of my books. (Imagine it’s like a subscription fee, but you also get one or two useful books on top!)

If you have already bought and read the books, please leave a review on the iBooks Store. Reviews are important to help new potential readers make the purchase decision. Thank you (again)!

And now Server.app, too!

There is a common understanding that celebrity deaths come in groups of three. Maybe Apple was aiming for that, too. After killing off Imaging and NetBoot/NetInstall, now there is a new support article:

Prepare for changes to macOS Server – Apple Support.

In this article Apple announces they will change the macOS Server app “to focus more on management of computers, devices, and storage on your network.” All other services will be deprecated.

The article lists the deprecated services and provides links to some open source alternatives.

  • Calendar
  • Contacts
  • DHCP
  • DNS
  • Mail
  • Messages (Jabber)
  • NetInstall (NetBoot)
  • VPN
  • Websites (Apache)
  • Wiki

In the beginning these services will remain available when you upgrade from an older version where they are activated, but will be hidden from new installations. In some unspecified future version of macOS Server, the services will be removed.

There are few services not listed here. They were already deprecated or moved to the ‘normal’ macOS in the last Server release. Open Directory and Software Update Server were deprecated and automatically hidden in Server 5.4 (the version which was released with macOS High Sierra). At the same time, Content Caching (Caching Server), File Sharing and Time Machine services moved from the Server app to the Sharing preference pane on macOS (and are available on every Mac, without having to purchase macOS Server). Xcode Server has moved into Xcode 9.

If you are using macOS Server for one of the above solutions, what should you do?

Don’t Panic

Apple is not killing off these services immediately. Server 5.5, which was released together with macOS 10.13.3 still has all the ‘normal’ services. Apple will hide the services in the UI to discourage their use in a future release. For the time being you can continue to use them. However, you need to start planning your move away from macOS Server.

While many Mac administrators would argue that macOS Server is not and never was a “professional” server, or even a server for any kind of deployment, it has found a niche in some small network environments. While the UI was certainly never perfect is has always been somewhat easier than messing with config files.

The replacements that Apple suggest in their article are worthy solutions if you need to maintain the services locally. Many are the open source projects that Apple used inside macOS Server themselves. While this removes the UI for monitoring and configuring the services, it also takes Apple out of the loop for updates and security patches. By getting the software directly you can get more timely updates. It also requires more maintenance and effort from the administrator, especially when you are using multiple services.

To the Cloud!

However, many of the above service are better replaced by cloud-hosted services, such as Office 365 or Google for Business/Education. These will also cover user identity management (replacing Open Directory) and file sharing with cloud storage systems.

For obvious reasons, DNS, DHCP and VPN cannot be run in the cloud. For small networks, these services are usually run on the router. However, if your router cannot run these services then you can run them on a dedicated box.

For my home network I am considering (i.e. finally found an excuse for) a Raspberry Pi.

NetBoot is still dead

Apple recommends NetSUS and BSDPy for NetBoot and NetInstall. These are certainly worthy solutions to host your nbi folders.

However, NetInstall functionality (this has been discussed before) is not present with the iMac Pro. It is to be expected that future new Mac hardware releases will follow the iMac Pro.

If you currently have a NetBoot/NetInstall based imaging or installation based workflow hosted on macOS Server, you need to be exploring alternative onboarding/setup workflows instead. DEP + MDM is the solution that Apple is pushing here.

Whatever solution you will find for your setup, it will require a lot of effort to get working smoothly. Rather than spending time and effort to move your NetBoot setup to BSDPy or NetSUS, leave it where they are for as long as they will still work and spend time on building a new supportable and supported workflow instead.

Whither macOS Server?

The Apple support article states:

macOS Server is changing to focus more on management of computers, devices, and storage on your network.

I would guess that ‘storage on your network’ means Xsan. Which some people still use. Seems weird to leave this as part of macOS Server and not split it out like other services. On the other hand it seems hard to imagine that this is some new server management feature.

What remains, is Profile Manager.

Profile Manager is considered Apple’s reference implementation of the MDM protocol. Most would not recommend using it in professional environments and few do (even fewer happily).

Now, that Apple is effectively reducing the functionality of macOS Server to Profile Manager, the question is: will it remain a mere reference implementation or will Apple finally put the resources behind Profile Manager to make it a usable, affordable and scalable solution?

Or maybe I get to write Profile Manager’s eulogy in a few years time as well. Only time will tell.

Does this mean Apple is leaving Enterprise business?

Really!? No.

In some ways Apple has never been able to enter Enterprise business with their own server products, hardware and software.

But they have been able to enter Enterprise business with their devices, Macs and iPhones and iPad. And because those devices are popular and trendy with Enterprise users, the Enterprises need to support them. That is what the MDM protocol and DEP are for.

With this step, Apple is making it clear that they are not even trying to play in the server business. They are happy to provide the MDM protocol and a reference implementation. They will support the infrastructure necessary to make DEP, MDM and VPP work. Apple is not interested in being the hardware that runs DNS, DHCP, file shares, Mail, calendaring and chat etc. Maybe not even the MDM server. Apple is very happy to leave this business to others. Apple sells devices.

macOS Server has been a neglected step child since the demise of the Xserve. I am surprised it took Apple this long to make it obvious.