Weekly News Summary for Admins — 2018-03-23

Expect some extra news next week as Apple has announced a keynote, in a school in Chicago, no less!

MacAD.UK videos are being posted on YouTube, including a presentation by yours truly…

If you would rather get the weekly newsletter by email, you can subscribe to the Scripting OS X Weekly Newsletter here!! (Same content, delivered to your Inbox once a week.)

#! On Scripting OS X

📰News and Opinion

🐞Bugs and Security

🔨Support and HowTos

Charles Edge has continued his posts on how to replace macOS Server:

Howard Oakley has been doing some interesting posts on the macOS unified log:

🤖Scripting and Automation

🍏Apple Support

♻Updates and Releases

📺To Watch

🎧To Listen

📚Support

I do not have any ads on my webpage or this newsletter. However, if you want to support me and this website, then please consider buying one (or both) of my books. (Imagine it’s like a subscription fee, but you also get one or two useful books on top!)

If you have already bought and read the books, please leave a review on the iBooks Store. Reviews are important to help new potential readers make the purchase decision. Thank you (again)!

Update 1.7 for ‘Packaging for Apple Administrators’

The advantage of self-publishing a digital book is that I can write and make updates. Some people have pointed out typos (thank you!) or had questions or comments about the book.

In some cases a question from a client or a MacAdmin Slack forum member will make me re-visit a section of the book and then add or re-write a paragraph or two.

The really great thing is that when you have already purchased the book, you will get the new improved update for free! And even if you have not bought the book yet, then you can know that when you buy it now, you will keep getting improvements in the future. It’s all just bits and bytes after all.

All of that said, the latest version of my book ‘Packages for Apple Administrators’ is now available on the iBooks Store.

I have fixed many typos and clarified a few sentences and paragraph. I have expanded and clarified the instructions for signing packages. There is also a new segment on the order of installation with distribution packages.

If you have already purchased books, all I ask in return for the free update, is to go to the iBooks Store and leave a review. The iBooks Store segregates reviews by territory, so every single one of them will be very important for other users to find and evaluate the book.

Thank you!

MacAD.UK Terminal Witchcraft and Wizardry Session

The fine people from MacAD.UK have started putting the sessions from this year’s conference online for every one to use.

The video of my presentation titled “The macOS School of Terminal Witchcraft and Wizardry” was uploaded earlier today. Go watch it!

You can find the slides and some notes on my session here.

 

Weekly News Summary for Admins — 2018-03-16

Charles Edge is dominating the summary today with a list of posts on how best to replace the waning macOS Server services. Thank you!

We also got the WWDC announcement from Apple. (June 4–8 in San Jose, CA) If you applied to the ticket lottery (before March 22, 10am PDT): Good Luck!

If you would rather get the weekly newsletter by email, you can subscribe to the Scripting OS X Weekly Newsletter here!! (Same content, delivered to your Inbox once a week.)

📰News and Opinion

🐞Bugs and Security

🔨Support and HowTos

🤖Scripting and Automation

♻️Updates and Releases

🎧To Listen

📚Support

I do not have any ads on my webpage or this newsletter. However, if you want to support me and this website, then please consider buying one (or both) of my books. (Imagine it’s like a subscription fee, but you also get one or two useful books on top!)

If you have already bought and read the books, please leave a review on the iBooks Store. Reviews are important to help new potential readers make the purchase decision. Thank you (again)!

Weekly News Summary for Admins — 2018-03-09

No big news this week, which is a nice change.

The first iPhone SDK dropped ten years ago this week. Craig Hockenberry wrote a wonderful retrospective which is worth reading.

If you would rather get the weekly newsletter by email, you can subscribe to the Scripting OS X Weekly Newsletter here!! (Same content, delivered to your Inbox once a week.)

📰News and Opinion

🐞Bugs and Security

🔨Support and HowTos

🤖Scripting and Automation

♻️Updates and Releases

📚Support

I do not have any ads on my webpage or this newsletter. However, if you want to support me and this website, then please consider buying one (or both) of my books. (Imagine it’s like a subscription fee, but you also get one or two useful books on top!)

If you have already bought and read the books, please leave a review on the iBooks Store. Reviews are important to help new potential readers make the purchase decision. Thank you (again)!

Weekly News Summary for Admins — 2018-03-02

A strange thing happened last Friday (a few hours after the last news summary). Apple updated the kbase article HT208020 – the infamous one that killed imaging. Then the article was reverted back to the old version after a few hours. Presumably some news for 10.13.4 were released prematurely.

Mark Melaccio grabbed screen shots.

If you would rather get the weekly newsletter by email, you can subscribe to the Scripting OS X Weekly Newsletter here!! (Same content, delivered to your Inbox once a week.)

#! On Scripting OS X

📰News and Opinion

MacAD.UK Session Links

(I have re-included the links from last week.)

🐞Bugs and Security

🔨Support and HowTos

🤖Scripting and Automation

🍏Apple Support

♻️Updates and Releases

📚Support

I do not have any ads on my webpage or this newsletter. However, if you want to support me and this website, then please consider buying one (or both) of my books. (Imagine it’s like a subscription fee, but you also get one or two useful books on top!)

If you have already bought and read the books, please leave a review on the iBooks Store. Reviews are important to help new potential readers make the purchase decision. Thank you (again)!

Single Brackets vs Double Brackets

In my recent post I mentioned in passing, that you should be using double brackets [[…]] for tests in bash instead of single brackets.

This is the post where I explain why. I also talked about this briefly in my MacSysAdmin session: Scripting Bash

Double Brackets are a bashism

Double brackets were originally introduced in ksh and later adopted by bash and other shells. To use double brackets your shebang should be #!/bin/bash not #!/bin/sh.

Since sh on macOS is bash pretending to be sh, double brackets will still work with the wrong shebang, but then your script might break on other platforms where a different shell might be pretending to be sh. Consistent behavior across platforms is the main point why sh is still around, so don’t use double brackets in sh (or use bash to use double brackets).

I go into detail on why to use bash over sh in this post: On the Shebang

Side note on syntax

In shell scripts you usually use tests in if or while clauses. These are tedious to write in the interactive shell. The ‘and’ operator && will execute the following statement only if the preceding statement returns 0 (success). So you can use && to write simple if … then … clauses in a single line.

if [ -d Documents ]
then
    echo "found docs"
fi

and

[ -d Documents ] && echo "found docs"

have the same effect. The second is much shorter, but as soon as the test or the command gets more complex you should revert to the longer syntax.

Alternatively, the ‘or’ operator || will only execute the following statement when the previous statement returns non-zero or fails:

[ -d Documents ] || echo "no docs"

is the same as

if [ ! -d Documents ]
then
    echo "no docs"
fi

What’s wrong with the single brackets?

The single bracket [ is actually a command. It has the same functionality as the test command, except that the last argument needs to be the closing square bracket ]

$ [ -d Documents && echo "found docs"
-bash: [: missing `]'
~ $ [ -d Documents ] && echo "found docs"
found docs
$ test -d Documents  && echo "found docs"
found docs

Note: in bash on macoS both test and [ are built-in commands, but as usual for built-in commands there are also executables /bin/test and /bin/[.

A single bracket test will fail when one of its arguments is empty and gets substituted to nothing:

$ a="abc"
$ b="xyz"
$ [ $a = $b ] || echo "unequal"
unequal
$ unset a
$ [ $a = $b ] || echo "unequal"
-bash: [: =: unary operator expected
unequal

You can prevent this error by quoting the variables (always a prudent solution).

$ [ "$a" = "$b" ] || echo "unequal"
unequal

Double brackets in bash are not a command but a part of the language syntax. This means they can react more tolerantly to ‘disappearing’ arguments:

$ [[ $a = $b ]] || echo "unequal"
unequal

You will also get an error if one of the arguments is substituted with a value with whitespace with single brackets, while double brackets can deal with this.

$ a="a"
$ b="a space"
$ [ $a = $b ] || echo "unequal"
-bash: [: too many arguments
unequal
$ [[ $a = $b ]] || echo "unequal"
unequal

Note: the = operator in sh and bash is for string comparison. To compare numerical values you need to use the -eq (equals), -ne (not equals), -gt (greater than), -ge (greater than or equal), -lt (less than), -le (less than or equal) operators. With double brackets you can also use two equals characters == for a more C like syntax. (or, better, use ((…)) syntax for arithmetic expressions)

Also, when using the = to assign variables, you cannot have spaces before and after the =, while the spaces are required for the comparison operator (both with single and double brackets):

a="a"           # no spaces
b="b"           # no spaces
[ "$a" = "$b" ] # spaces!
[[ $a = $b ]]   # spaces!

Since the single bracket is a command, many characters it uses for its arguments need to be escaped to work properly:

$ [ ( "$a" = "$b" ) -o ( "$a" = "$c" ) ]
-bash: syntax error near unexpected token `"$a"'
$ [ \( "$a" = "$b" \) -o \( "$a" = "$c" \) ]

You could alternatively split this example into two tests: [ "$a" = "$b" ] || [ "$a" = "$c" ].

Double brackets interpret these characters properly. You can also use the (again more C like) && and || operators instead of -a and -o.

 [[ ( $a = $b ) || ( $a = $c ) ]]

In general, you can work around most of the issues with single bracket syntax, but the double bracket syntax is more straight forward and hence more legible and easier to type.

Double bracket features

Aside from the cleaner syntax, there are a few ‘bonus’ features you gain with double brackets.

With double brackets you can compare to * and ? wildcards, and bracket globbing […]:

$ a="Documents"
$ [[ $a = D* ]] && echo match
match
$ a=hat
$ [[ $a = ?at ]] && echo match
match
$ [[ $a = [chrp]at ]] && echo match
match

You can also use < and > to compare strings lexicographically:

$ a=cat
$ b=hat
$ [[ $a < $b ]] && echo sorted
sorted

And you get an operator =~ for regular expressions:

$ a=cat
$ b="the cat in the hat"
$ [[ $a =~ ^.at ]] && echo match
match
$ [[ $b =~ ^.at ]] && echo match

Note that you should not quote the globbing patterns or the regex pattern.

Summary

  • you should use bash for shell scripting on macOS
  • when using bash, you should use double brackets instead of single brackets
  • double brackets are safer, easier to type and read, and also add few neat features

defaults – the Plist Killer

Last week, fellow MacAdmin Kyle Crawford has discovered that on macOS High Sierra the defaults will delete a property list file with invalid plist/XML syntax, even when you just attempt to read data. Erik Holtham has a more detailed OpenRadar bug.

Patrik Wardle has found the relevant code, which deletes the file when it doesn’t validate.

This is bad. Thanks to all involved for finding, documenting and sharing this.

This is new behavior in High Sierra. I am not yet sure which version of High Sierra this new behavior was introduced. The behavior makes sense in the context of an application attempting to read a settings file, but the defaults tool deleting arbitrary files is, of course, dangerous.

Update: This behavior has been fixed in 10.13.4. However, it is still good practice to avoid defaults for anything other than actual preferences files.

What to do?

As usual, don’t panic. This will only affect Macs running High Sierra and corrupt or broken files. However, if you have a script that accidently points the defaults command at a different file, it will delete that. So you have to use it with care.

It is probably a good practice to verify a file before you attempt to modify it with the defaults command in a script with the plutil command:

if ! plutil -lint path/to/file.plist; then
    echo "broken plist"
    exit 1
else
    defaults path/to/file …
fi

Alternatives to defaults

Alternatively, you can and should use plutil or PlistBuddy to read and modify property list files.

Learn more about plutil, PlistBuddy and other tools to read and write property lists in my book: “Property Lists, Preferences and Profiles for Apple Administrators”

plutil is unfortunately not really useful to read a single value from a property list file. The extract verb will show any value as its own plist. The plutil command is useful to edit existing plist files. (Read details on the plutil command.)

PlistBuddy, however is very useful for both reading an writing values to a property list file:

$ /usr/libexec/PlistBuddy berry.plist -c "print size"
$ /usr/libexec/PlistBuddy berry.plist -c "set size enormous"

PlistBuddy has the additional advantage of allowing to add or edit values nested deep in dict or array structures.

You can get more information on PlistBuddy in its man page or my book.

The interactive mode of PlistBuddy is also very useful.

So the defaults command is dead?

No.

Apple has been warning us to not use defaults for generic property list file editing and parsing for quite a while in the defaults command’s man page:

WARNING: The defaults command will be changed in an upcoming major release to only operate on preferences domains. General plist manipulation utilities will be folded into a different command-line program.

As this warning states, the defaults tool reads and writes data to plist files through macOS’s preferences system. This has the advantage that the tool gets (and changes) the current value whether it is cached in memory or not. When an application is listening for notifications that a preference has changed (not many do) then it will be notified.

Files for preference domains are usually stored in /Library/Preferences/, ~/Library/Preferences or their ByHost subfolders. Sandboxed applications will have their preference plist files in their container.

There, however, many other files that are property list files which are not part of the user defaults system: launchd files, configuration profiles and AutoPkg` recipes to name just a few.

Mac Admins commonly use the defaults tool, despite Apple’s warning, to create, read and edit generic plist files. As mentioned above, plutil, PlistBuddy or direct manipulation through Obj-C, Python or Swift, are better choices for generic plist files.

You can learn about all these options in my book: “Property Lists, Preferences and Profiles for Apple Administrators”

Weekly News Summary for Admins — 2018-02-23

This week I was travelling to London for MacAD.UK. So maybe I missed some interesting links. If I did, please let me know and I will include the link next week.

MacAD.UK was great. I had a great time meeting many people, some I already knew and some I had only met virtually so far. Great presentations, hope to see you all again next year!

If you would rather get the weekly newsletter by email, you can subscribe to the Scripting OS X Weekly Newsletter here!! (Same content, delivered to your Inbox once a week.)

#! On Scripting OS X

📰News and Opinion

MacAD.UK Session Links

Let me know if I missed one!

🐞Bugs and Security

🔨Support and HowTos

🤖Scripting and Automation

🍏Apple Support

♻️Updates and Releases

🎧To Listen

📚Support

I do not have any ads on my webpage or this newsletter. However, if you want to support me and this website, then please consider buying one (or both) of my books. (Imagine it’s like a subscription fee, but you also get one or two useful books on top!)

If you have already bought and read the books, please leave a review on the iBooks Store. Reviews are important to help new potential readers make the purchase decision. Thank you (again)!