Is Packaging Dead?

I wrote a book on Packaging, you may have heard. (Please buy it! If you have bought and read it, please leave a review!)

I have been asked whether it was wise to publish a book on Packaging now. There is lots of talk in the Mac Admin community about the future of admin tools. The general consensus is that Apple wants to move macOS further towards a locked down system like iOS (and watchOS and tvOS). We have already seen the first moves in that direction with System Integrity Protection. Some suspect the new file system APFS may break traditional imaging workflows.

Note: I used to work for Apple, but that was years ago. I have no inside knowledge and all this is speculation, based on my many years of experience working with and for the company. I hope I am right, but would be the first to admit that I have made very wrong predictions regarding Apple before.

Up to now, these changes are not too severe. Some (myself included) even approve of deprecating legacy practices such as imaging, given that the alternative solutions bring many other benefits. However, there is the spectre of the completely locked down macOS, which can only be managed the same as iOS can: with an MDM server.

I do not think this is Apple’s ultimate goal. I also believe it will take longer than many are afraid of, and it may be even impossible in the long term without breaking the nature of macOS fundamentally. I’d like to give a few examples:

Omnigroup has been a productive and successful developer for macOS since NeXTSTEP. They were on Mac OS X before there was a Mac OS X. Last year, the OmniGroup switched some of their applications in the Mac AppStore from the one-time purchase model to a free-download with trial and in-app purchase model. I understand why Omnigroup decided to do this. However, Apple has no VPP solution for in-App-Purchases so an administrator cannot deploy Omni software with an MDM anymore. OmniGroup still sells their apps directly, so you can buy a volume deployment license directly from OmniGroup and deploy without the Mac AppStore, VPP, or MDM.

Veertu is an interesting new virtualisation solution, which started out in the Mac App Store. It could do that because it uses Apple’s Hypervisor Framework, which is meant to provide the system resources necessary for virtual machines to SandBoxed apps. However, last year, Veertu removed their solution from the Mac App Store because the limitations were still too stringent. Similarly, VMware Fusion, Parallels and VirtualBox are all not in the Mac App Store and not deployable with VPP or MDM.

MATLAB is a powerful solution for building scripts and tools with mathematical and engineering libraries. Calling MATLAB an ‘app’ would be like calling your MacBook a ‘typewriter with a screen.’ Scientists and engineers not only use MATLAB to build complex mathematical and statistical solutions, they share those solutions as products or open source tools. Many people may not even be aware that they are using a tool running the MATLAB runtime in the background. MATLAB is one of many powerful math/numerical/statistics tools, such as Wolfram Mathematica, IBM SPSS, R, etc. None of which are in the Mac App Store or even possible with the current Mac App Store limitations.

These math tools are deeply entwined with the UNIX philosophy of tools passing data from one to the other to do specialised steps of processing. Some workflows I have encountered are Rube-Goldberg-Machine-like sequences of MATLAB, Python, and bash scripts with a Qt or Java UI. As monstrous as these ‘solutions’ may be, they are essential for many people’s day-to-day work.

The only Adobe application in the Mac App Store is Photoshop Elements. There are many more Adobe apps in the iOS App Store, but none of their core Creative Cloud applications. One of the reasons may be that many Mac users, not only rely on the applications, but on plug-ins for these applications.

For some solutions, like AutoCad, the plug-ins have plug-ins. There is no method to provide, sell and deploy plug-ins for applications in the Mac App Store. You can argue that App Extensions can fill that functionality, and may even be more flexible than app-specific plug-ins, but they have certainly not been adopted on a large scale yet.

While macOS has improved how users discover and install drivers for printers, there is no way of managing driver deployment in the MDM spec. Administrators still have to deal with downloading and deploying printer driver installers and settings. The same is true for any other hardware driver software. While you can argue that the vendors sometimes seem to go out of their way to make driver deployment as complex as possible, even the best of them have no way of providing their software in the Mac App Store or manage it with an MDM.

This is but a short list of examples. I am certain any admin can easily list a many more. All of them have in common that Mac administrators need to use package installers and/or shell scripts to install and manage these solutions. Sometimes you can re-use the vendor installer, but even then you need to know how to inspect and verify these files. In many cases you have to reverse engineer the installation and re-build your own packages. (How to do all of this is covered in my book: “Packaging for Apple Administrators”)

As long as the Mac App Store has no manageable (VPP) solutions for upgrades, subscriptions and in-App-Purchases, it has to be considered broken for client management. Any such feature which gets added to the Mac App Store (such as subscriptions last year) have to have a VPP solution, otherwise it is useless for managed deployments.

The Mac App Store excludes entire categories of software because of the sandbox requirement, software providers. Users and admins still need a way to deploy that software. If Apple were to go ahead and lock down macOS completely, users and admins would have to either jailbreak their Macs, or abandon the Mac platform.

The number of users who would be affected by this is paltry, compared to the hundreds of millions of iOS devices Apple sells each year. So, the cynic will, “what does Apple care if a few hundred thousand users leave?”

A locked down macOS would provide little benefit over iOS running on an iPad Pro. A locked down MacBook may have more processing power (barely), and more ports for connecting hardware, but the software to use them to their full extent would simply not be available.

Apple wants to lock down macOS to make it more secure, because that distinguishes macOS from the other desktop systems. On the other hand, Apple has an interest in keeping macOS open because it is not iOS.

Apple has its own set of Mac applications that have no iOS counterpart: Xcode, Logic Pro X and Final Cut Pro. These are the ‘Pro’ apps, the lighter apps: GarageBand and iMovie have iOS counterparts. Swift Playgrounds which may be considered ‘Xcode lite’ is iOS only.

There are good reasons for each of these not being on iOS. Final Cut Pro and Logic Pro X rely very much on external hardware for input, most of which is not available for iOS. Xcode relies on integration with many command line tools (llvm, git, etc.) to develop its full potential. There may be some projects within Apple that can be built without shell scripts, but I am sure they are in the minority. Just look at the postinstall scripts of any Apple installer package.

Interesting Sidenote: Xcode, GarageBand and Logic Pro bypass the Mac App Store limitations by installing additional components at first run. Apple is hitting the limitations of their own rules. There is also no way of deploying these additional components with an MDM.

There are many people at Apple who use their Macs for graphic design, hardware design, running complex computations, writing shell scripts, compiling complex software and hardware drivers, and testing on virtual machines, etc. Do you believe Apple wants to buy them Dells so they can run Photoshop, bash, AutoCAD, SPSS, or VMware? Apple also needs to deploy and manage the Macs in their retail stores. They are, after all, one of the largest deployments of their own OS.

So, assuming Apple is not planning to lock down macOS like iOS, what are they going to do?

They certainly will continue to restrict access by users and processes to security critical parts of the OS. They also want to gain (more) control of vectors where malicious software can inject itself into the system. Apple obviously wants to use MDM as the main conduit for Macs to be managed. The MDM spec already has the ability to deploy an custom package to a client. This is probably supposed to push client agents for management systems for features not included in the MDM spec (such as reporting).

Apple, third party MDM vendors and open source MDMs could build on this existing feature to provide package installation of software for non-App Store solutions. This alone would alleviate many concerns that administrators currently have. Maybe the packages will have to be signed with an Apple developer certificate or some other extra security step. (Learn how sign packages in my book.) This will require major re-architecting of many tools such as Munki, but those are do-able, maybe even desirable. The format the MDM spec is using to install software is, not surprisingly, a flat package installation file, so packaging skills will be needed more, rather than less in this scenario.

From my perspective, Apple’s choices are to obsolete the Mac platform, or to gradually keep improving security and working on the Mac App Store, VPP, DEP and MDM as management solutions while maintaining the openness that makes the Mac such a powerful and creative tool.

While I believe that ‘pro’ users of the Mac platform need to get accustomed to the fact that they are not Apple’s main focus anymore, I also refuse to believe that Apple is planning to obsolete the Mac entirely, because they would hurt themselves in the process.

The balance between security and openness will not be easy to find. The openness is what sets the Mac platform apart from iOS. The control and security is what sets macOS apart from the other desktop operating systems. There will be much back and forth between the ‘pro’ users and Apple. We as admins have to keep harping and nagging at Apple to do the right thing. (File bugs!)

The macOS open Command

Most Terminal users will know that

$ open .

will open the current working directory in a Finder window. (You, dear wonderful reader, know this because you read my previous post on Terminal-Finder Interaction.)

However, the open command can do so much more.

Folders

Trivially, it cannot merely open the current working directory, but any path:

$ open ~/Library/Preferences
$ open /etc
$ open ../..

This can be used as a quick way to navigate to hidden directories.

You can also open multiple folders at once:

$ open ~/Documents ~/Desktop ~/Downloads
$ open ~/D*

To clean up, you can option-click any close button in a Finder window to close all Finder windows. Or you can use the keyboard short cut ⌘⌥W.

Files

open can also open files. In general you can think of open as the command line equivalent of double-clicking a file or folder in Finder.

$ open document.pdf

will open document.pdf in the current working directory with the default application for PDF files (usually Preview). You can use this against multiple files as well:

$ open ~/Desktop/Screen\ Shot\ *.png

will open all screenshot files (if any) in a viewer in the default application (Preview).

Applications

If you have changed the default application that handles a file type or want to override the default application, you can use the -a option:

$ open -a Preview ~/Desktop/Screen\ Shot\ *.png
$ open -a TextEdit web.html

You can specify just the name of an application or the full path, i.e. /Applications/Preview.app. If you need to be specific, you can also specify an application’s bundle identifier with -b com.apple.Preview.

If you want to open a document but keep the application and the new document window in the background, use the -g option.

$ open -g ~/Desktop/Screen\ Shot\ *.png

Text Editors

There are two interesting special cases for designating applications:

$ open -e helloworld.swift

will open a file with TextEdit.

$ open -t helloworld.swift

will open a file with the default application for text files (.txt file extensions) You can use the Finder Info panel to change the default application or, if you want more fine grained control use RCDefaultApp. In the default macOS config these are the same, but you can of course change the default app to your favourite text editor. (Many text editors, like BBEdit and Atom, have their own CLI tool, but if they don’t, you can use open -t instead.)

You can even pipe text into open with the -f option:

$ ls -l ~ | open -f     # TextEdit, '-e' is implied
$ ls -l ~ | open -tf    # default application assigned to txt

You can set your $EDITOR environment variable: EDITOR='open -tnW'; export EDITOR and then command lines tools that expect text from an editor, like git commit, will get the text from open and thus your default text editor instead. The -n option will actually open a new (sometimes second) instance of the application and the command line tool will resume when you quit this new instance. This a somewhat awkward workflow for Mac users. Many text editors provide a command line tool that may work better in these cases. For BBEdit the correct $EDITOR value is bbedit -w --resume.

Showing Files in Finder

If you are working on a file in Terminal and want to locate it in Finder, open can do better than just opening the enclosing folder. It can select a given file as well:

$ open -R helloworld.swift

Will open a Finder window with the enclosing folder of helloworld.swift and select the file. (You can pass multiple files into open -R but it will only select the last file in the list.)

URLs

Finally there is one more useful thing you can open:

$ open http://scriptingosx.com   # default browser
$ open vnc://TestMac.local       # Screen Sharing
$ open x-man-page://open         # show man page in Terminal

and, as always, you can use the -a option to override the default application:

$ open -a Firefox http://scriptingosx.com

Header files

For the sake of being complete: you can also open header files quickly with open. The -h option will search and open the header file for a given class. There is an additional -s option to choose an SDK:

$ open -h NSTask
$ open -h NSTask -s OSX10.12
$ open -h UIView.h -s iPhoneOS10.2
$ open -a BBEdit -h NSTask

If the search term is ambiguous open will list all the options.

Terminal–Finder Interaction

Mac Admins have to work a lot in Terminal. This seems counter-intuitive for an OS that is famed for its user interaction. I can’t talk for all admins, but for me the strength of macOS/OS X was always in the combination of ‘clicky’ UI and command line. When you know what you are doing, you can get the best of both worlds.

I remember an Apple marketing slogan: “The power of Unix, the simplicity of Mac” This is from OS X Lion, so more than five years old by now. The future will show how long Apple still values the ‘powerful’ Unix underpinnings. But for now, they are still available and I am going to use them.

All of that said, the CLI and the UI are not entirely separate areas in macOS, there is a lot of overlap and there are functions in Finder and Terminal that allow for quick interaction between the two.

Finder to Terminal

You can drag any folder from Finder to the Terminal application icon in the dock and Terminal will open a new window and change the working directory to the folder you dragged.

Movie 1: Drag Folder onto Terminal

You can drag the folder icon from the Finder window title bar, as well.

Movie 2: Drag Folder from Title Bar onto Terminal

When you drag any file into an open Terminal window, it will insert the full path to that file or folder with spaces and other special characters properly escaped.

Movie 3: Drag File onto Terminal

You can drag multiple files and Terminal will insert all of their paths, separated by spaces. For example you can type file[space] in Terminal and then drag multiple files into that window and hit return, to get information on the file type.

Movie 4: Drag Multiple Files onto Terminal

If you prefer, you can get the same effect with copy and paste. Just select the files in Finder, choose ‘Copy’ (⌘C), switch to Terminal and ‘Paste’ (⌘V).

Update: I knew I had forgotten one. Thanks to Elliot Jordan who pointed this one out on Twitter:

Dragging a folder into a Terminal window while holding the command (⌘) key will add cd before the path to a folder. When you command-drag a file it will cd to the enclosing folder of that file.

Movie 5: Command Drag a Folder to Terminal

Getting Finder path from Terminal

If you are already in Terminal and want to get the frontmost Finder window, we have to do some homework first. (I got the idea for this script from this post, though I have modified its solution somewhat.) This command

$ osascript -e 'tell app "Finder" to get posix path of ((target of window 1) as alias)'

will give us the correct path, but it has two downsides: a) it is awfully complex to type repeatedly and b) it fails with an error if no Finder window is open.

To avoid typing this long command every time, we have two options. You can either define the command as a function in your .bash_profile (or the respective profile for your preferred shell) or you can save it as a script in your $PATH.

To define it as a function, add this to the .bash_profile:

# prints the path of the front Finder window. Desktop if no window open
function pwdf () {
    osascript <<EOS
        tell application "Finder"
            if (count of Finder windows) is 0 then
                set dir to (desktop as alias)
            else
                set dir to ((target of Finder window 1) as alias)
            end if
            return POSIX path of dir
        end tell
EOS
}

# changes directory to frontmost Finder window
alias cdf='pwdf; cd "$(pwdf)"'

The pwdf function will just print the path to the frontmost Finder window to stdout.

There is also added an alias to quickly change directories to the frontmost Finder window. cdf will also print the path it is changing to, like the cd - command. (Which changes to the previous working directory.) This has to be defined as an alias, since scripts cannot change a shell’s working directory.

If you prefer to define the pwdf command from a script file use the following code:

Save this as a text file without extension in a folder in your $PATH, and set the executable bit with chmod +x /path/to/pwdf. You also need to remove (or comment) the function from your profile, since that would override the script.

Using the script form of osascript allows us to pass arguments into the AppleScript. (You can do that with function as well, but the syntax gets really messy quickly.) This script will list the paths to all open Finder windows with the -a|--all argument. Also when you provide a string as an argument it will search for a Finder window containing that string:

$ pwdf Pref
/Users/armin/Library/Preferences/

The open command

You can also go from Terminal to the Finder. This usually as simple as typing

$ open .

This will open the current working directory ‘.’ in a Finder window.

This is usually where most online ‘hints’ for the open command start and end. However, open has so much more to offer. So much more, in fact, that I will cover the open command in a separate article.

Update for ‘Packaging for Apple Administrators’

I have pushed an update to the Packaging for Apple Administrators book in the iBooks Store. Among a few typos and minor changes, I have added an Appendix section on legacy package formats.

If you already bought the book, you should be able to download the updated content for free. If you buy it now then you will get this update and any future ones!

If you have read and liked the book, please leave a review on the iBooks Store, I would be very grateful!

Get it on iBooks

Mac Manager Meeting – Packaging Presentation – Notes and Links

These are the links for my presentation at the Marriot Library of the University of Utah. You should soon be able to watch an archived version of the presentation here.

Thanks again for letting me speak and for watching and listening!

Slides for the Presentation (as PDF)

Packaging is Dead?

Packaging Tools

AutoPkg

Packaging Book

Mac Managers Meeting Presentation

I will be presenting on packaging Wednesday, Jan 18th (update, I had the wrong date here earlier) at the Mac Managers Meeting of the University of Utah. It will be at 1pm Mountain Time (21:00 Central European). Unfortunately, I will not be present but doing the presentation remotely. There will be a live stream and the talk will be archived to view later.

There will be a brief introduction to some basic pkgbuild and autopkg functions and a more detailed look at the new trust functionality in autopkg 1.0. I hope there will remain time for some Q&A.