Category: Tools

Distribution packages are a special format of installer packages. Distribution packages can contain one or more normal or component packages. They can also contain extra resources to customize and control the user interface in the Installer application.

In most cases administrators prefer component packages since they are easier to create and maintain. However, there are a few cases where distribution packages are necessary:

• add a package to a custom installation in NetInstall, AutoDMG or a system installer created with createOSXinstallPkg
• combine multiple component pkgs into a single installer
• restrict hardware and system requirements
• modify the interface presented in Installer.app
• push the package with MDM’s InstallApplication command

Building Distribution Packages

You can easily convert an existing component package, built with pkgbuild to a distribution package with the productbuild command:

$ productbuild --package component.pkg dist.pkg

You can also combine multiple components into a single distribution package:

$ productbuild --package A.pkg --package B.pkg combined.pkg

You can add the --sign option to the productbuild command when the distribution package needs to be signed:

$ productbuild --sign "Installer: Armin" --package component.pkg dist.pkg

You can find valid identities with

$ security find-identity -p basic -v

The string you pass with the --sign parameter can be a partial match to the full identity name.

Note: munkipkg has a flag to build a distribitution package instead of a component package.

Extracting Component Installers from Distribution Packages

Sometimes you may want to extract a component installer pkg from a distribution package.

First you need to expand the distribution pkg with pkgutil:

$ pkgutil --expand dist.pkg dist_expanded

When you use the --expand option on a distribution package, components will also be expanded into subfolders that end in .pkg. Because of this Finder will erroneously display them as installer bundle files. This is misleading, since the components are not functional in this form.

When you want to use the component package without any modifications, you can quickly recompress or ‘flatten’ the expanded component:

$ pkgutil --flatten dist_expanded/component.pkg component.pkg

The process of expanding and flattening a component will of course remove any signature the original pkg might have had. You can re-sign the flattened package with productsign:

$ productsign --sign "Installer: Armin" component.pkg component_signed.pkg

Note: Obviously, when you are tearing a distribution package apart you need to know what you are doing. Components in a distribution package may depend on other components or on scripts and tools in other components. As always: test, test, test.

Packaging Book

You can learn more on building installer packages in my book: “Packaging for Apple Administrators”

Earlier this year I wrote a post on whether packaging is dead. Since I wrote a book on Packaging and have also invested much of my career in macOS I do have quite some interest in the topic.

(Please buy the book! If you have bought and read it, please leave a review!)

After that post I made myself a reminder to re-visit the topic post WWDC. I was very much expecting to be proven wrong or hopelessly optimistic. This reminder has been bugging me for a while. I have had a hard time to consolidate my thoughts into writing.

It’s not that this year’s WWDC was boring. Quite the opposite. The new iPads Pro look wonderful and I want one. Apple also announced great new iMacs and MacBooks and a space-grey iMac Pro, demonstrating they still care about the Mac line. (The Mac mini, however, got no love this time around. I do hope the line gets at least a speed bump and we don’t have to wait for a the new Mac Pro to get a decent option for screenless Macs. I’ve given up on servers…) And finally, both iOS 11 and macOS High Sierra (10.13) look like solid updates with lots of new features for users and developers. This was a great WWDC!

Mac admins were concerned that this update would lock down macOS in a similar fashion to iOS. The worst case scenarios painted a picture where not even admin users would be able to get root privileges and you couldn’t install third party daemons and agents any more, fundamentally breaking the way all management systems work. Admins would have to re-work their workflows to work through MDMs, which are not yet capable to bear this burden. The new Apple File System APFS would break NetBoot and all the tools admins use to image Macs.

What happened was… well… nothing much really.

Mac-narök has been postponed.

(Excellent talk by Micheal Lynn at MacDevOps YVR, just a few minutes before the WWDC Keynote. Go watch it.)

There will be changes in High Sierra that affect admins. APFS on macOS is definitely going to happen. In the current (first) beta there is an option to disable the filesystem conversion during upgrade, but it is unknown wether that option will still exist in the release. You can now add iOS devices to DEP even if they were not registered at purchase. You can control a firmware password on Macs with profiles. There are some (minor) changes to files and folders protected by SIP.

I don’t believe or want to suggest the posts above and many other people who predicted the end of Mac Administration as we know it were hysterical or unnecessarily panicked. When they were written there were strong indications and hints that Apple was planning a lockdown of some form soon. MDM only Mac administration might still happen in a future update. However, we seem to have gotten an reprieve, which is good.

Why did the lockdown not happen now? Excellent question, which I do not know the answer to. There was a big outcry from the Mac admin community and many used their official channels (Apple reps and support, Radar, Feedback) to tell Apple what a huge imposition such a quick and drastic change would be. Also many third-party application developers are reluctant to (or cannot) move to the Mac App Store, which would be a requirement in an MDM only world.

For now it seems that common admin tools will run on High Sierra and APFS with just some minor adjustments. This includes packages! Packaging is not dead! Long live Packaging! …and all the other tools!

(On the other hand, some things may still break or be removed during the beta phase.)

Does that mean we should just happily keep doing what we are doing? No. Even if Apple does not yet enforce ‘MDM-only’ they are clearly moving towards ‘more MDM.’ We still have to re-evaluate every setting and workflow with MDM in mind. There are some great solutions already that can combine MDM with e.g. Munki, Chef or Puppet.

Even though imaging, whether you choose the “thick” or “thin” approach, will probably still work in High Sierra, you should be thinking about alternative strategies. DEP plus application installs and updates are more flexible and powerful than full disk imaging.

There are certain setups, such as classrooms and training centers, which require frequent re-imaging with short turnaround times. Ironically, the tech that was predicted to kill imaging might provide a solution. APFS disk snapshots could provide a solution for fast system restores. The tools for this do not seem to be fully in place yet, but the time to test and file bugs is now.

The MDM ‘InstallApplication’ command, which installs the agent software, such as the Munki or Jamf client, should be supported by management systems. This would allow clients to be connected to the management system without user interference and the client software to add to the limited functionality of MDM with tools that admins already have solutions and expertise for.

So the post WWDC summary: the ‘End of Things as We Know Them’ has been postponed. Imaging will still work, but you want to start examining and testing alternatives. Packages and scripts remain relevant, but there are interesting new means of distributing them.

It is already apparent the next WWDC will have more exciting news ready for Mac and iOS Admins. Until then we will be busy learning the new features and tools in High Sierra and iOS 11 and laying the groundwork to the future.

In an earlier post we talked about how to append to the PATH variable, so you can add your own directories to bash’s search path.

In macOS the default PATH on a ‘clean’ installation is:

$ echo $PATH
/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin:

However, if you have installed some tools (such as the macOS Server.app, Xquartz or Munki) you will see those in the PATH as well:

$ echo $PATH
/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin:/opt/X11/bin:/Applications/Server.app/Contents/ServerRoot/usr/bin:/Applications/Server.app/Contents/ServerRoot/usr/sbin:/usr/local/munki:

Where does this pre-set PATH come from?

Since the PATH is pre-set on a clean new account without a .bashrc or .bash_profile, we have to look in a central location which applies to all users. In another earlier post, we saw that /etc/profile is run for every user shell, even before a .bash_profile is executed. When you look into this file, you see that the very first set of commands look like this:

if [ -x /usr/libexec/path_helper ]; then
    eval `/usr/libexec/path_helper -s`
fi

This looks very promising. The path_helper tool has a man page. This tool does a few things to assemble the PATH. First it reads the file /etc/paths which on macOS looks like this:

/usr/local/bin
/usr/bin
/bin
/usr/sbin
/sbin

So this is where the default ‘clean’ macOS PATH comes from. Then path_helper will read every file from /etc/paths.d and append each line of each file in that directory to the PATH as well. This is where optional and third party applications and tools, like Xquartz or Munki, can install their own additions to the PATH for all users.

(Files in this folder will be read in alpha-numerical order of the filename. Some tools, like Xquartz, attempt to influence the order by preprending a number, e.g. 40-XQuartz.)

Finally, if path_helper runs in an environment where PATH is already set, it will append that PATH value to what it built from the files and then remove duplicates.

path_helper does not change the environment variable directly, but it generates the commands necessary to set the PATH correctly. It will generate the right commands wether it is called from a bourne type shell (on macOS: sh, bash, ksh and zsh) or a csh type shell (on macOS: csh and tcsh). You can see the output of the two styles by running path_helper with the -s or -c options:

$ /usr/libexec/path_helper -s
PATH="/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin"; export PATH;
$ /usr/libexec/path_helper -c
setenv PATH "/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin";

(You have to type the full path to path_helper because, ironically, but intentionally, /usr/libexec is not in the standard PATH.)

To actually execute the commands generated by path_helper you can use the eval command, like the /etc/profile does:

$ eval $(/usr/libexec/path_helper)

Don’t touch my Profile!

Some command line tool installers understandably feel the need to add their tools to the default PATH. Since there is no unified approach among different flavors of UNIX and Linux on how to do this, you will find several different approaches. Some tools will edit /etc/profile and others will look for the various profile files in a user’s home directory and edit those. Usually the installation process will append a line that appends their tools directory to the PATH.

One example for this is the Python 3 installer. It contains a compnent package that will attempt to determine which profile file you are using and appends a line to append to the PATH.

However, this is not only highly intrusive but also quite fragile. Changes to /etc/profile might be overwritten by a future macOS update. Changes to a user’s profile file, might be overwritten by the user. Also the installer will only append their setting to the current user, not other users that may be present or created in the system.

Sample paths.d installer package

On the other hand, dropping a file into /etc/paths.d with a package installer will affect all users on a system. The file in paths.d can be updated for future updates if necessary and is also easily identified and removed by an admin. It will work wether it is installed to the startup volume or another volume. It can be pushed with management tools.

Building an installer for a file in /etc/paths.d is very simple:

$ mkdir CustomToolPathInstaller
$ cd CustomToolPathInstaller
$ mkdir payload
$ echo "/usr/local/customtool" >> payload/customtool
$ pkgbuild --root payload --install-location /private/etc/paths.d --version 1.0  --identifier com.example.customtool.path CustomToolPath.pkg
pkgbuild: Inferring bundle components from contents of payload
pkgbuild: Wrote package to CustomToolPath.pkg

Only five commands, three of which create the folder structure. You can find this sample project (which is slightly more elaborate) on my GitHub.

If you want to learn more about building installer packages for macOS, please read my book “Packaging for Apple Administrators”.

What about MANPATH?

This is usually not used on macOS since the the default settings for the man tool are quite flexibel. (Look at the man page for man and the file /etc/man.conf for details.) However, if a MANPATH environment variable is set when path_helper runs, it will also assemble the command to set the MANPATH built in a similar way to the PATH from the files /etc/manpaths and the directory /etc/manpaths.d.

Usually the MANPATH is not set on macOS so you will not see this. But if you want to manage your MANPATH and want to leverage path_helper all you have to do is set the MANPATH.

$ export MANPATH=/usr/share/man
$ /usr/libexec/path_helper
PATH="/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin"; export PATH;
MANPATH="/usr/share/man:/usr/local/share/man"; export MANPATH;

Re-order the PATH

We have seen path_helper is extremely useful. There is one caveat, however. path_helper may reorder your PATH. Imagine you are pre-pending ~/bin to your PATH because you want to override some standard tools with your own. (Dangerous, but let’s assume you know what you are doing.) Then some process launches a subshell which can call path_helper again. path_helper will ‘find’ your additions to the defined PATH, but it will append to the list of default paths from /etc/paths and /etc/paths.d, changing your order and thus which tools will be used.

$ export PATH=~/bin:$PATH
$ echo $PATH
/Users/armin/bin:/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin
$ /usr/libexec/path_helper 
PATH="/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin:/Users/armin/bin"; export PATH;

You can see behavior like this when you use Xterm (The X11 based terminal in Xquartz) which does not execute .bash_profile but still picks up the PATHenvironment variable from somewhere…

# in Xquartz Terminal:
bash-3.2$ echo $PATH
/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin:/opt/X11/bin:/Applications/Server.app/Contents/ServerRoot/usr/bin:/Applications/Server.app/Contents/ServerRoot/usr/sbin:/usr/local/munki:/Library/Frameworks/Python.framework/Versions/3.5/bin:/Users/armin/bin:/opt/X11/bin

A better way to override built-in commands which is not affected by path_helper would be to use bash aliases or functions in your profile.