Random Desktop Background Color with desktoppr

File this one under: probably useless, but fun.

I recently updated desktoppr with the feature to control the scaling of a custom desktop picture. Because a scaled desktop picture might not cover the entire screen, macOS also allows you to choose a custom color to fill the remaining area. desktoppr v0.3 can be used to control both of these settings.

So, I thought I could use this to just set a random single color as the desktop background, similar to how I did it for Terminal windows.

macOS does not allow you to set no desktop picture. So I created a PNG file that is only a transparent background. This image is basically an invisible desktop picture and all you see on screen is the fill color. Then, you can set any fill color with desktoppr color.

I have added a randombackgroundcolor script to the examples in the desktoppr repository on GitHub.

It’s a bit silly, but also kind of fun. Enjoy!

Advanced Quoting in Shell Scripts

Quoting strings and variable substitutions is a bit of a dark art in shell scripts. It looks simple and straightforward enough, but there are lots of small devils in the details, that can come out and haunt you.

Basics: why we quote strings

In shell scripts (sh, bash, and zsh) you use the equals character = to assign a string value to a variable:

> name=John
> dirpath=/Library

As long as there are no special characters in the literal string, there is no need to quote the string.

When you use the variable, you prefix a $ symbol:

> echo $name
John
> cd $dirpath
> pwd
/Library

When the literal string contains special characters, you need to either escape the special characters with the backslash \ or quote the entire string with either single quotes ' or double quotes ". Space is proverbial ‘killer character’, especially for file paths. (More details in this post.)

name='John Doe'
dirpath="/Library/Application Support"

The difference between single quotes and double quotes is important. Single quotes escape every special character except the single quote itself. A single quoted string of '#$"\!' will represent exactly those characters.

Double quotes escape most characters, except the double quote " the backtick `, the dollar sign $, the backslash \, and the exclamation mark !. (There are slight differences between the shells on this.)

This allows us to use old-style command substitution with backticks and variable substitution (dollar sign) within double quoted strings:

> echo "Hello, $name"
Hello, John Doe
> echo "The Computer Name is `scutil --get ComputerName`"

Though you should be using the $(…) syntax for command substitution instead of backticks `. The parenthesis syntax is more readable and can be nested.

In general, it is a good rule to always quote literal strings. Whether you should use double quotes or single quotes depends on the use case.

Combining literal strings with special characters

Things can start getting complicated when you want special characters with their special functionality. For example, when you want to refer to the path ~/Library/Application Support, you should put it in quotes, because of the space. But when you put the ~ in the quotes, it will not be substituted to the user’s home directory path.

There are a few ways to solve this problem. You could escape the space with a backslash. You could use the $HOME variable instead (but be sure you are in a context where this is set). But the easiest is to move the special character out of the quotes:

dirpath=~"/Library/Application Support"

Quotes in quotes

Sometimes it is necessary to have a set of quotes within quotes. A common situation for MacAdmins is the following osascript:

osascript -e 'display dialog "Hello, World"'

The osascript command can be used to run Apple commands or scripts. Since AppleScript uses double quotes for literal strings, the entire AppleScript command is passed in single quotes. This keep the command string together and the double quotes in single quotes don’t confuse the shell.

This works fine, until you want to do something like this:

computerName=$(scutil --get ComputerName)
newName=$(osascript -e 'text returned of (display dialog "Enter Computer Name" default answer "$computerName")')

Again, we put the AppleScript command in single quotes, so we can use double quotes inside. But now, the single quotes are also blocking the variable substitution and we get the literal $computerName in the dialog.

There are a few solutions out of this, I will demonstrate three:

First, you could close the single quotes before the variable substitution and re-open them after:

osascript -e 'text returned of (display dialog "Enter Computer Name" default answer "'$computerName'")'

This will in this form as long as $computerName contains no spaces. This is unlikely as the default computer name is something like Armin's MacBook Pro. The shell will consider that space a separator before a new argument, breaking the AppleScript command into meaningless pieces and failing the osascript command. We can avoid that by putting the substitution itself in double quotes:

osascript -e 'text returned of (display dialog "Enter Computer Name" default answer "'"$computerName"'")'

This works and is entirely legal syntax, but not very legible.

Escaping the escape characters

Another solution is to use double quotes for the entire AppleScript command, we can use variable substitution inside. But then we have to deal with the double quotes required for the AppleScript string literal. The good news here is that we can escape those with the backslash:

osascript -e "text returned of (display dialog \"Enter Computer Name\" default answer \"$computerName\")"

This doesn’t win prizes for legibility either, but I consider it an improvement over the previous approach.

Here Docs

The above approaches with work in sh, bash, and zsh. But bash and zsh have another tool available that can work here. The ‘here doc’ syntax can be used to include an entire block of AppleScript code in a bash or zsh script:

#!/bin/bash

computerName=$(scutil --get ComputerName)

newName=$(osascript <<EndOfScript
    text returned of (display dialog "Enter Computer Name" default answer "$computerName")
EndOfScript
)

echo "New name: $newName"

The syntax is a bit weird. The <<EndOfScript says: take all the text until the next appearance of EndOfScript and pipe it into the preceding command, in this case osascript.

The ‘marker’ EndOfScript is entirely arbitrary. Many people choose EOF but I prefer something a little more descriptive. Whatever label you choose the ending marker has to stand alone in its line. This is why the parenthesis ) which closes the command substition $( has to stand alone in the next line.

You can still use variable substitution in a here doc, so the variable $computerName will be substituted before the here doc is piped into osascript.

macOS shell command to create a new Terminal Window

Of course, you can easily create a new Terminal window from the ‘Shell’ menu or by using the ⌘N (or ⌘T) keyboard shortcut. But in some cases, it can be more useful to use a shell command.

New windows created with the keyboard shortcut or from the menu will always have the home directory ~ as the current working directory. What I want, is a new window that defaults to current working directory or a custom directory that I can provide with an argument:

> new           # opens a new terminal window at the current working directory
> new ~/Desktop # opens a new terminal window at ~/Desktop

No luck with AppleScript

After my last success using AppleScript, I thought this would be the best solution again. Unfortunately, this particular piece of the AppleScript dictionary is broken. The make new window or make new tab commands fail with errors and I have tried several combinations.

After some web searching, it looks like this has been broken for a long time. I filed an issue in Feedback Assistant.

You can create a new Terminal window with AppleScript using the do script command in the Terminal dictionary. (Not to be confused with do shell script.) So this AppleScript, sort of does what I want, but seems cumbersome.

tell application "Terminal"
    do script "cd ~/Desktop"
end tell

If you know of a better way to create a new Terminal window or, even better, a Terminal tab with AppleScript, then please let me know. (No UI Scripting solutions – those have their own issues.) I have a few other ideas where this might come in useful.

Enter the open command

During those web searches, I also found suggestions to use the open command, instead:

> open -a Terminal ~/Documents

Will open a new Terminal window with ~/Documents as the working directory. This is already really close to what I wanted.

I created this function in my shell configuration file (bash, zsh):

# creates a new terminal window
function new() {
    if [[ $# -eq 0 ]]; then
        open -a "Terminal" "$PWD"
    else
        open -a "Terminal" "$@"
    fi
}

With this, I can now type

> new Projects/desktoppr

and get a new Terminal window there. This is very useful when combined with the history substitution variable !$ (last argument of previous command):

> mkdir Projects/great_new_tool
> new !$

And an unexpected, but useful side effect is that the new function can also open an ssh session in a new window:

> new ssh://username@computer.example.com

Hope you find this useful, too!

Book Update – Moving to zsh v3

I have pushed an update for the “Moving to zsh” book.

Just a few changes and fixes that have accumulated over the past two weeks. Much of this has been from feedback of readers. Thanks to everyone who sent in their notes.

The update is free if you have already purchased the book. You should get a notification from the Books application to update. (On macOS, I have seen that it can help to delete the local download of the book to force the update.)

If you are enjoying the book, please rate it on the Books store, or (even better) leave a review. These really help, thank you!

Also, please recommend the book to friends, co-workers, and anyone else (not just MacAdmins) who might be facing the zsh transition as they upgrade to Catalina.

The changes in v3 are listed here. This list is also in the ‘Version History’ section in the book. There, you will get links to the relevant section of the book, so you can find the changes quickly.

  • Added a section explaining how to work with upper- or lower-case strings in zsh scripts
  • Added a section explaining the differences in the read built-in command
  • Clarified the section on Connected Variables
  • Fixed file names in the table for Configuration Files and added a note for how to use configuration files with python environments
  • As usual, several typos and clarifications (Thanks to many readers)

Random Terminal Background Colors

In an older post, I showed a trick to get random terminal backgrounds from a selection of colors. Others have used similar approaches.

While I have used this for a long time, the limited number of colors has always annoyed me. And with the introduction of Dark Mode in macOS it seemed just not useful anymore.

Mike’s approach to create the color files with a python script sent me down a rabbit hole to recreate this in Swift. I actually succeeded in creating such a Swift tool, but then, when I worked on connecting the tool with Terminal, I found an even simpler, and arguably better way to do this.

Surprisingly, it involved AppleScript.

Changing the Terminal Background color

Terminal has a powerful AppleScript library, which allows to read and change (among other things) the background color of a Terminal window or tab:

tell application "Terminal"
    get background color of selected tab of window 1
            --> {65535, 65533, 65534}
end tell

The background color is returned as a list of three RGB numbers ranging from 0 to 65535 (216 – 1). You can also set the background color:

tell application "Terminal"
    set background color of selected tab of window 1 to {60000, 45000, 45000}
end tell

This will set the background color of the current window to pastel pink (salmon?).

Armed with this knowledge it is fairly straight forward to write a script that will set the background color to a random color:

#!/usr/bin/osascript

on backgroundcolor()
    set maxValue to (2 ^ 16) - 1
    
    set redValue to random number from 0 to maxValue
    set greenValue to random number from 0 to maxValue
    set blueValue to random number from 0 to maxValue
    
    return {redValue, greenValue, blueValue}
end backgroundcolor

set newcolor to backgroundcolor()

tell application "Terminal"
    set the background color of the selected tab of window 1 to newcolor
end tell

You can paste this code in Script Editor and hit the ‘play’ button and watch the Terminal window change. But since we want to use from within Terminal, we will take a different approach: paste the code into your favorite text editor and save it as a text file named randombackground (no extension).

Then open Terminal and change directory to where you save the file and set its executable bit:

> chmod +x randombackground

Now you can run this AppleScript file like any other script file from Terminal:

> ./randombackground

This is fun!

I am not the first to discover and use this, Daniel Jalkut and Erik Barzeski have documented this in 2006.

Enter Dark Mode

Fast forward back to 2018: Along with the rest of macOS, Terminal gained “Dark Mode” in macOS Mojave.

The default “Basic” window profile in Terminal has black text on a white background in light mode and white text on a black background in dark mode. There is some “magic” that happens when the system switches to Dark or Light mode.

However, once we customize the background color (or any other color) the magic does not work any more. When our random backgrounds are too dark in light mode (or vice versa), they don’t really look nice any more, and the text becomes hard to read or completely illegible.

So, we want to change the script to detect dark or light mode and limit the colors accordingly. You can detect dark mode in AppleScript with:

tell application "System Events"
  get dark mode of appearance preferences
end tell

This will return true for dark mode and false for light mode. We modify the script to use just a subrange of all available colors, depending on the mode:

#!/usr/bin/osascript

on backgroundcolor()
    set maxValue to (2 ^ 16) - 1
    tell application "System Events"
        set mode to dark mode of appearance preferences
    end tell
    
    if mode then
        set rangestart to 0
        set rangeend to (maxValue * 0.4)
    else
        set rangestart to (maxValue * 0.6)
        set rangeend to maxValue
    end if
    
    set redValue to random number from rangestart to rangeend
    set greenValue to random number from rangestart to rangeend
    set blueValue to random number from rangestart to rangeend
    
    return {redValue, greenValue, blueValue}
end backgroundcolor

set newcolor to backgroundcolor()

tell application "Terminal"
    set the background color of the selected tab of window 1 to newcolor
end tell

When you run this from Terminal for the first time, it may prompt you to allow access to send events to “System Events.” Click ‘OK’ to confirm that:

Automatically setting the background color

Now we can randomize the color by running the command. For simplicity, you may want to put the script file somewhere in your PATH. I put mine in ~/bin/, a folder which a few useful tools and I also added to my PATH(in bash and in zsh).

It is still annoying that it doesn’t happen automatically when we create a new window or tab, but that is exactly what the shell configuration files are for. Add this code to your bash or zsh configuration file.

# random background color
if [[ $TERM_PROGRAM == "Apple_Terminal" ]]; then
    if [[ -x ~/bin/randombackground ]]; then
        ~/bin/randombackground
    fi
fi

Our script will likely fail when the shell is run in any other terminal application or context (such as over ssh). The first if clause checks if the shell is running in Terminal.app. Then the code check to see if the script exists and is executable, and then it executes the script.

This will result in random Terminal colors, matching your choice of dark or light mode.

Note: macOS Catalina added the option to automatically switch the theme depending on the time of day. This script will detect the mode correctly when creating a new window, but Terminal windows and tabs that are already open will retain their color. I am working on a solution…

Upper- or lower-casing strings in bash and zsh

String comparison in most programming languages is case-sensitive. That means that the string 'A' and 'a' are considered different. Humans usually don’t think that way, so there is bound to be trouble and confusion.

If you are looking at single letters, the bracket expansion can be quite useful:

case $input in
    [aA])
        # handle 'a'
        ;;   
    [bB])   
        # handle 'b'
        ;;
    [qQ])
        # handle 'q': quit
        exit 0
        ;;
    *)  
        echo "Option is not available. Please try again"
        ;;
esac

However, for longer strings the bracket expansion gets cumbersome. To cover all case combinations of the word cat you need [cC][aA][tT]. For longer, or unknown strings, it is easier to convert a string to uppercase (all capitals) or lower case before comparing.

sh and bash3

Bash3 and sh have no built-in means to convert case of a string, but you can use the tr tool:

name="John Doe"
# sh
echo $(echo "$name" |  tr '[:upper:]' '[:lower:]' )
john doe

# bash3
echo $(tr '[:upper:]' '[:lower:]' <<< "$name")
john doe

Switch the [:upper:] and [:lower:] arguments to convert to upper case.

There are many other tools available that can provide this functionality, such as awk or sed.

Bash 5

Bash 5 has a special parameter expansion for upper- and lowercasing strings:

name="John Doe"
echo ${name,,}
john doe
echo ${name^^}
JOHN DOE

Zsh

In zsh you can use expansion modifiers:

% name="John Doe"
% echo ${name:l}
john doe
% echo ${name:u}
JOHN DOE

You can also use expansion flags:

% name="John Doe"    
% echo ${(L)name}     
john doe
% echo ${(U)name}
JOHN DOE

In zsh you can even declare a variable as inherently lower case or upper case. This will not affect the contents of the variable, but it will automatically be lower- or uppercased on expansion:

% typeset -l name      
% name="John Doe"
% echo $name        
john doe
% typeset -u name      
% echo $name        
JOHN DOE

Comparing Version strings in zsh

Another excerpt from the book “Moving to zsh.” I found this one so useful, I thought I’d like to share it.

You can get the version of zsh with the ZSH_VERSION variable:

% echo $ZSH_VERSION
5.7.1

And you can get the version of macOS with the sw_vers command:

% sw_vers -productVersion
10.15.1

Comparing version strings is usually fraught with potential errors. Strings are compared by the character code for each character.‘2’ is alphabetically greater than ‘10’ when compared as strings, because the character code for 2 is greater than the character code for 1. So, a string comparison of macOS version numbers will return that 10.9.5 is greater than 10.15.1.

Zsh, however, provides a function is-at-least which helps with version string comparisons.

With a single argument, is-at-least will return if the current zsh version matches or is higher than a given number:

if ! is-at-least 2.6-17; then
  echo "is-at-least is not available"
fi

When you provide two arguments to is-at-least, then the second argument is compared (using version string rules) with the first and needs to match or be higher:

autoload is-at-least
if is-at-least 10.9 $(sw_vers -productVersion); then
  echo "can run Catalina installer"
else
  echo "cannot run Catalina installer"
fi

Note: when used in a script, you will probably have to autoload is-at-least before using it. In an interactive shell, it is often already loaded, because many other autoloader functions will have already loaded it.

Install shellcheck binary on macOS (updated)

A few months back I wrote a post on how to compile and build an installer for the shellcheck binary for macOS.

Just a few weeks later, the shellcheck project added a download for a pre-compiled binary for macOS. You can now download the binary with this link:

https://shellcheck.storage.googleapis.com/shellcheck-latest.darwin.x86_64.tar.xz

Ironically, macOS can unarchive xz archives when you double click them in the Finder, but there is no command line tool on macOS to unarchive them. In the previous post, I ran into the same problem, and there you can find instructions on how to install the xz tools on macOS.

Update 2020-03-26: I was unnecessarily complicating this. You can use tar to unarchive this:

tar -xf shellcheck-latest.darwin.x86_64.tar.xz

After downloading and un-archiving, you can manually move the shellcheck binary to a suitable directory. The standard location is /usr/local/bin.

For manual installations, this is it! Much simpler than before. Thank you!

Note: if you want the man page as well, you still need to build it with pandoc from the source.

Build a pkg for managed deployment

If you are a MacAdmin and want to distribute shellcheck with your management system, you will need to build an installer package (pkg).

Instead of copying the binary to /usr/local/bin, place it in a payload folder in a project folder. Then build the pkg with pkgbuild:

% mkdir -p ShellcheckPkg/payload
% cp ~/Downloads/shellcheck-latest/shellcheck ShellcheckPkg/payload
% pkgbuild --root ShellcheckPkg/payload --identifier com.example.shellcheck --version 0.7.0 --install-location /usr/local/bin shellcheck-0.7.0.pkg

Replace the 0.7.0 with the actual version number.

Automated Package creation with autopkg

And because all of this isn’t really that difficult, I built autopkg recipes for Shellcheck You can find them in my recipe repository or with autopkg search shellcheck. Enjoy!

Associative arrays in zsh

This is an excerpt from my book “Moving to zsh” which is available for order on the Apple Books Store.

One of the advantages of zsh over bash 3 is the support of “associative arrays,” a data structure known as hash tables or dictionaries in other languages.

In associative arrays, you can store a piece of data, or value with an identifying ‘key’. For example, the associative array userinfo has multiple values, each identified with a key:

% echo $userinfo[name]
armin
% echo $userinfo[shell]
bash
% echo $userinfo[website]
scriptingosx.com

Note: bash 4 also added associative arrays, but they are implemented slightly differently.

Creating associative arrays

In zsh, before you can use a variable as an associative array, you have to declare it as one with

declare -A userinfo

This will tell the shell that the userinfo variable is an associative array. You can also use typeset -A as an alternative syntax. You can verify the type of the variable:

% echo ${(t)userinfo}
association

You can then set the key-value pairs of the userinfo associative array individually:

userinfo[name]="armin"
userinfo[shell]=bash
userinfo[website]="scriptingosx.com"

When you set the value for an existing key again, it will overwrite the existing value:

% echo $userinfo[shell]
bash
% userinfo[shell]=zsh
% echo $userinfo[shell]
zsh

Setting the values for each key is useful in some situations, but can be tedious. You can also set the entire associative array at once. There are two syntaxes for this in zsh:

userinfo=( name armin shell zsh website scriptingosx.com )

This format follows the format ( key1 value1 key2 value2 ...). The other syntax is more verbose and expressive:

userinfo=( [name]=armin [shell]=zsh [website]="scriptingosx.com" )

When you set the associative array variable this way, you are overwriting the entire array. For example, if you set the userinfo for ‘armin’ like above and then set it later like this, the website key and value pair will have been overwritten as well:

% userinfo=( [name]=beth [shell]=zsh )
% if [[ -z $userinfo[website] ]]; then echo no value; fi
no value

If you want to partially overwrite an existing associative array, while leaving the other key/value pairs intact, you can use the += operator:

% userinfo+=( [shell]=fish [website]=fishshell.com )
% echo $userinfo[name]                                           
beth
% echo $userinfo[shell]
fish
% echo $userinfo[website]
fishshell.com

To clear an associative array, you can use:

% userinfo=( )

Retrieving data from an associative array

We have already seen you can get the value for a given key with the ‘subscript’ notation:

% echo $userinfo[name]                                           
beth

When you access the $userinfo variable directly, you will get a normal array of the value:

% echo $userinfo
beth fish fishshell.com

You can also get an array of the keys with this syntax:

% echo ${(k)userinfo}
name shell website

or a list of both keys and values:

% echo ${(kv)userinfo} 
website fishshell.com shell fish name beth

You can use this to copy the data from one associative array to another:

% declare -A otherinfo
% otherinfo=( ${(kv)userinfo )
% echo $otherinfo[name]
beth

You can also use this to loop through all the keys and values of an associated array:

for key value in ${(kv)userinfo}; do
    echo "$key -> $value"
done

#output
website -> fishshell.com
shell -> fish
name -> beth

Limitations

Associative arrays have their uses, but are not as powerful as dictionaries in more powerful languages. In zsh, you cannot nest associative arrays in normal arrays, which limits their use for complex data structures.

There is also no functionality to transfer certain file formats, like XML or property lists directly in to associative arrays or back.

Shell scripting was never designed for complex data structures. When you encounter these limitations, you should move “up” to a higher level language, such as Python or Swift.

Download a full ‘Install macOS’ app with softwareupdate in Catalina

Catalina is out! While you are preparing for your upgrade, here’s a nice new feature for MacAdmins:

The softwareupdate command has gained a new option in Catalina:

% softwareupdate --fetch-full-installer

Will download the latest ‘Install macOS’ application to this Mac’s /Applications folder. This is extremely useful for many admin tasks.

The --fetch-full-installer flag has a sub-flag: --full-installer-version which allows you to download a specific version.

% softwareupdate --fetch-full-installer --full-installer-version 10.14.6

During my testing in the Catalina beta version I was able to download 10.15, 10.14.6, 10.14.5, and 10.13.6. I was not able to test if 10.13.6 would download the hardware specific build of 10.13.6 for the 2018 MacBook Pro, since I do not have that hardware.

I would assume that downloading an Installer application for a macOS version that is not supported on the hardware you are running the command on would fail. (Again I did not have such hardware available for testing.)

So far the only way to download the macOS Installer in some automated fashion was using Greg Neagle’s installinstallmacos.py script. That script still has some abilities that do not seem to be available to the softwareupdate command, but it is good to see Apple accepting the need for this kind of workflow.