Terminal: The ‘[‘ Marks the Spot

Using Terminal on macOS you may at some point have wondered about the small gray square brackets before the prompt. (There is also a counterpart closing bracket ‘]’ at the right side of Terminal window.)

Marks in Terminal

They appeared first in Terminal in El Capitan (10.11). They are called ‘Marks’ and simplify scrolling through the output in the Terminal window.

By default, every command that is executed automatically gets marked.

Jump to the Mark

You can quickly scroll the Terminal output to previous marks (i.e. command prompts) with ⌘↑ (Command-Up Arrow) and to next mark with ⌘↓ (Command Down-Arrow). This allow you to quickly skip through the command prompts in your window.

Select Output

You can quickly select output between Marks. ⌘⇧↑ (Command-Shift-Up) will select the output up to, but not including the previous command prompt. ⌘⇧↓ (Command-Shift-Down) will select the output up to, but not including the next command prompt. When you repeatedly hit this key combo the selection will be extended further, including the intermittent command prompts.

⌘⇧A (Command-Shift-A) or ‘Select Between Marks’ from the ‘Edit’ menu, will select to the previous mark, if you are the last (empty) command prompt, otherwise it will select to the next mark.

Clearing Output

You can also just remove the output of the previous command with ‘Clear to Previous Mark’ from the ‘Edit’ menu (⌘L). This is less destructive than clearing the entire screen or buffer.

Marks and Bookmarks

Marks are very useful. However, they are all the same, so jumping back to a specific mark in your Terminal scroll buffer will get harder and harder you have to hit ⌘↑ more and more often.

You can elevate a mark to a bookmark, which will allow you to jump directly to it. Jump to the line you want to bookmark or select some text and choose ‘Mark as Bookmark’ from the ‘Edit’ menu (⌘⌥U) or context menu.

Bookmarks have a bolder vertical bar ‘|’ at the left and right edge of the window.

You can jump to previous and next bookmarks with ⌘⌥↑ and ⌘⌥↓, or you jump directly to a bookmark from the list in the ‘Edit’ > ‘Bookmark’ menu.

The ‘Edit’ > ‘Bookmarks’ menu also has ‘Insert Bookmark’ and ‘Insert Bookmark with Name…’ items. These will mark the current command prompt as a bookmark, and give you the option of naming the bookmark, rather than having the default timestamp as the name.

Terminal will set an automatic bookmark when restoring a Terminal window (after a reboot). This setting is controlled in the ‘Resume’ area in the ‘Window’ Tab of the Profiles area of the settings.

Not Leaving Marks

If the grey brackets indicating the marks annoy you, you can hide them with ‘Hide Marks’ from the ‘View’ menu. This will only hide the marks and bookmarks, they will still be automatically (or manually) created and you can still jump to them.

You can disable automatic mark creation entirely by deselecting ‘Automatically Mark Prompt Lines’ from the ‘Edit’ > ‘Marks’ submenu. When you have disabled automatic marking, you can execute and manually mark a prompt with ⌘↩︎ (Command-Return).

With automatic marking enabled, you can execute a command without marking with ⌘⇧↩︎ (Command-Shift-Return).

You can also manually remove a mark, by scrolling or jumping to it and choosing ‘Unmark’ from the ‘Edit’ > ‘Marks’ menu or the context menu.

Marking Spots Programatically

I have added the following aliases to my .bash_profile:

alias mark="osascript -e 'if app \"Terminal\" is frontmost then tell app \"System Events\" to keystroke \"u\" using command down'"
alias bookmark="osascript -e 'if app \"Terminal\" is frontmost then tell app \"System Events\" to keystroke \"M\" using command down'"

This way I can put the mark in between two commands and just let them run:

$ system_profiler SPApplicationsDataType; mark; system_profiler SPExtensionsDataType

The downside of this approach that the aliases do not work when Terminal is in the background.

Apple acquires Workflow app

You’ve probable heard it already: Apple has bought the Workflow application. The developers will keep working at Apple and the Workflow app is now offered for free on the AppStore, although with a few interesting changes.

Workflow has always been one of those apps that would have liked to get into. However, as an admin, my workflows center on macOS and Unix. You cannot build packages on iOS. Even iBooks Author only exists on macOS. As much as I like the iPad and iOS it has been relegated to secondary device because of the focus of my work.

Still it will be interesting to see how Apple will integrate Workflow with iOS and the Apple applications. I am looking forward to it.

If you now want to catch up with what Workflow is all about, you can’t do much better than Frederico Viticci’s articles on MacStories.

Sierra update 10.12.4: Disable iCloud Desktop and Documents Sync

Babo D: Disable iCloud Desktop and Documents Sync

Very useful discovery. When you search Apple’s profile documentation page for ‘10.12.4’ you also find a new key to disable Touch ID login on the new MacBooks and an entire new payload type for SmartCard setup.

BTW: you can of course learn more about configuration profiles in my new book: ‘Property Lists, Preferences and Profiles for Apple Administrators’ available in the iBooks Store now!

PR3 Book Launch Gift! — Command Line Reference

‘Property Lists, Preferences and Profiles for Apple Adminstrators’ launches today! Go get it before the iBooks Store runs out of bytes!

You can read more about the book in my pre-sale announcement post.

While writing the book. I built a list of common and important commands for the ‘Property Lists, Preferences and Profiles for Apple Administrators’ book. (Thanks to François for the idea!)

The list is included in the book resources as a PDF, so you can print it and hang it above your desk. However, you can also download it directly here.

(If you are a consultant, trainer or other service provider of some sorts, your customers might enjoy these, too. There is even white space on the second page for your contact and logo.)

Enjoy!

And when you finish the book, please leave a review on the iBooks Store!

Update to ‘Packaging for Apple Administrators’

Aside from the new book ‘Property Lists, Preferences and Profiles for Apple Administrators’ (available for pre-sale right now, goes on sale Monday) my first book also got some love. ‘Packaging for Apple Administrators’ got a new cover, to better fit with the cover for PR3 and also a few new internal additions.

iBooks should have already notified owners of the update. If you do not own it yet, go get it on the iBooks Store!

If you like the book, please leave a review!

Terminal and the Clipboard

Continuing in my informal series of Terminal articles, I’d like to visit two tools that help interact from the shell to a particular part of the macOS UI: the clipboard.

The clipboard is the ‘place’ where macOS stores anything you cut (⌘X) or copy (⌘C). Later the system reads from the clipboard (and possibly converts the data) when you paste (⌘V).

Terminal can trivially interact with the clipboard. You can select text in the Terminal and copy it, and then paste it elsewhere, you can also paste text in the command prompt itself.

(This is useful when you read a great iBook with many Terminal commands. You can simply copy the command from the iBook text and paste into Terminal.)

There are however, two commands, specific to macOS that connect the clipboard closer to other shell commands.

pbcopy will take the contents of stdin (usually text, but could be any stream of data) and put them in the clipboard. So, anything you pipe into pbcopy will end up in the clipboard, so you can paste it into a different application.

(NeXTSTEP was the operating system that Mac OS X was originally based on. What was called the ‘clipboard’ in Mac OS was called ‘pasteboard’ in NeXTSTEP. While all references visible to the user where changed to the Mac OS naming, you still find the old NeXTSTEP names in the ‘innards’ of macOS, hence pbcopy and pbpaste.)

Examples:

# easier than open, select all, copy
$ cat ideas.txt | pbcopy
# converts Word Doc to plain text and puts it in the clipboard
$ textutil -convert txt MyBook.docx -stdout | pbcopy
$ ipconfig getifaddr en0 | pbcopy
$ system_profiler SPHardwareDataType SPSoftwareDataType | pbcopy
$ uuidgen | pbcopy

I use the last command a lot when I need UUIDs for custom configuration profiles. It is much easier to pipe the output directly into the clipboard, than to select and copy the output.

One downside of this it, that you cannot see what is piped into the clipboard. You can easily make the new clipboard contents visible by typing pbpaste as the next command:

$ uuidgen | pbcopy
$ pbpaste
A524B454-5B42-4832-943D-896DF755FDEC

or

$ uuidgen | pbcopy; pbpaste
95DC9C0E-052E-4896-A4D3-1BB5EAECD93C

(Several UUIDs were wasted writing this article.)

pbpaste is the counterpart to pbcopy. pbpaste will take the contents of the clipboard, and if they are plain text write them to stdout. That alone can be useful when you want to visualize the clipboard but gets more powerful when you pipe or substitute it into other unix commands.

Examples:

# after copying the output of uuidgen
$ plutil -replace PayloadUUID -string $(pbpaste) MyConfigProfile.mobileconfig
# copy an html snippet from somewhere or
$ echo '<a href="http://scriptingosx.com">Scripting OS X</a>' | pbcopy
# then
$ pbpaste | textutil -stdin -format html -convert rtf -stdout | pbcopy

This last command will convert an HTML snippet into rich text (rtf).

New Book — Property Lists, Preferences and Profiles

As it happens so often, the plan changed.

The plan was to go from ‘Packaging’ right to the obvious sequel ‘Automated Packaging’. And I am working on that book. However, since AutoPkg uses property list files for the recipes, I needed to introduce property list file formats and all the tools, quirks and caveats.

So the sidenote on property lists grew into an appendix. And it kept growing. Then I thought it would be a waste to explain property list files but not talk about preference files. And once you explained preference files, you should also introduce configuration profiles, especially custom profiles. Profiles are property lists after all.

So I moved the chapter on property lists into a new iBook. And I added chapters on preferences and configuration profiles, leading to a cumbersome, but memorable title. (‘PR3’ for short.)

So, here we are, not a sequel, but a ‘sidequel’. I will of course continue writing ‘Automated Packaging’ once this book is published.

PR3 is not quite done yet. I am still getting great feedback from the proofreaders. (Thank you!) But all the small pieces that are still missing or need refinement should fall in to place next week and I aim to release on March 20. (Deep breaths!)

And there will be updates to the iBook (like with ‘Packaging’) as I find more typos and learn more in future.

As with ‘Packaging’ the main target audience is either an administrator who is new to the Mac platform or a Mac user who is new to Mac administration and management. However, the proofreaders have all told me, there are many nuggets and pieces of new information for experienced admins as well.

Until then you can download the first chapter on property list file formats from the iBooks Store!

You can also pre-order and get the book automatically, as as soon as I press the ‘Publish’ button!

Get it on iBooks

Overall, writing these books is a great experience. It is a great chance to dive deeply into a topic. It is also a great chance to give back to the community that is amazingly generous in sharing knowledge and experience. There are many more topics on my list that I would like to write books on.

On Bash History Substitution

There are some things in every profession or area of expertise that seem so obvious to you and most people around you. However, there was a point in your career where you did not know this yet and somebody taught you.

A brief discussion on the MacAdmins Slack yesterday led to such a moment. A bash shortcut that seems so obvious to me and probably to many who read this, but again, not everyone knows. And it was someone’s lucky ten thousand moment to learn an wonderful shell shortcut:

sudo !!

Re-visit your (shell) History

The bash shell remembers a certain number of commands that you entered. This way you can ‘recall’ them with the up arrow or the ctrl-R keystroke.1

There are also history substitutions to save typing. And not only do they save typing but these automations, like tab-completion will make you type less errors.

Sudo Make me a Sandwich2

So imagine you typed

$ profiles -P
profiles: this command requires root privileges

and the system helpfully reminds you that you need root privileges to run this. You could retype the entire command after sudo. You could also hit up-arrow, ctrl-a and type sudo[space][return], which is some improvement.

Or you just type

$ sudo !!
sudo profiles -P
Password:
There are no configuration profiles installed

The shell will substitute the !! (‘bangbang’) with the previous command. The shell will print the entire command after the substitution (this can be really helpful when things don’t work as expected) and then immediately execute the full command.3

Personally, I only ever use this with sudo. I could imagine there are other uses, maybe with ssh user@host -c.

You can also just type !! to repeat the last command, I find ‘up-arrow, return’ to be more intuitive (and one keystroke less).

Historical Arguments

The other history substitution I use is more specific, but still has its uses. The shell substitutes !$ with the last used argument. The catch here is that if the previous command had multiple arguments, !$ will be the very last argument on that line, not all the arguments.

Some examples:

$ mkdir -p MyNewInstaller/payload
$ cd !$
cd MyNewInstaller/payload
$ touch postinstall
$ chmod +x !$
chmod +x postinstall
$ pkgbuild --component /Volumes/Firefox/Firefox.app --install-location /Applications Firefox-51.0.1.pkg
$ cp !$ /Volumes/SoftwareArchive/Mozilla/
cp Firefox-51.0.1.pkg /Volumes/SoftwareArchive/Mozilla/

(Most pkgbuild examples are easily dated with the Firefox version.)

If the last command had no arguments, !$ will be the ‘zeroth’ argument, the command itself:

$ sw_vers
ProductName:    Mac OS X
ProductVersion: 10.12.3
BuildVersion:   16D32
$ !$
sw_vers
ProductName:    Mac OS X
ProductVersion: 10.12.3
BuildVersion:   16D32

More History

There are more bash history substitutions. I found a helpful StackOverflow post with examples here. Most are a bit too cumbersome to actually use the frequently.

Though, I myself had a ‘lucky ten thousand’ moment, since I did not know about !* (all the previous command’s arguments) and !^ (the previous command’s first argument) until today.

Path Through History

This is not directly related to the shell history, but `cd` also remembers your previous location in the file system and you can quickly jump back with `cd -`.

$ cd ~/Projects
$ cd ~/Library/AutoPkg/
$ cd -
/Users/armin/Projects
$ pwd
/Users/armin/Projects


  1. hit ctrl-R and type some text, to get the first command in your history containing that text
  2. If you do not recognise that reference, you are one the ten thousand, again.
  3. Yes, technically, sudo !![return] is the same number of keystrokes as up-arrow, ctrl-a and sudo[space][return] but I still find it easier.