Show the git status in the Terminal Window Title

I mentioned in my post on Customizing the zsh prompt, that you can get certain information from the git repository into the prompt or right prompt.

This subject has grown into a book: reworked and expanded with more detail and topics. Like my other books, I plan to update and add to it after release as well, keeping it relevant and useful. You can order it on the Apple Books Store now.

That post contains an example adapted from the ‘Pro git’ documentation which shows how to display the current branch and repo in the prompt.

Personally, I don’t like the Terminal window to be cluttered up with repeated information. While the information from the git repository does change over time, it does not change that frequently, and the information gets repeated over and over.

While I was researching the last post, which describes how to display the current working directory in Terminal’s window title bar, I learnt that you can also set a window title using a different escape code.

So, instead of repeating the git information on every prompt, we can show it in the window or tab title bar.

You can find the code to set this up in my dotfiles repository on Github. To use it make sure the git_vcs_setup file is in your fpath and load it in your .zshrc with

# Git status
autoload -U git_vcs_setup && git_vcs_setup

(More info on fpath and autoloading functions.)

Note that this file sets up two functions: update_terminal_window_title and update_terminal_tab_title. Terminal can have separate information titles for tabs and the entire window. When there are no tabs (or technically, just a single tab) Terminal displays both titles.

The status info above will show the repository, the current branch, a U when there are unstaged changes, and a S when there are staged, but uncommitted changes.

If you want more details in the status, you might want to consider using a more powerful solution to retrieve the git status, such as this project.

Weekly News Summary for Admins — 2019-09-20

Apple’s release schedule this year is certainly not following any previous pattern.

After releasing a beta of iOS 13.1 during the beta phase of 13.0. Apple then announced that iOS 13.0 (for the iPhones) would be released on Sep 19 (yesterday) and iOS 13.1, together with iPadOS 13.1 on September 30. Yesterday, iOS 13.0 was released on schedule, but the release date for iOS and iPadOS 13.1 was moved ahead to September 24.

watchOS 6 was release yesterday alongside iOS 13, but only for Series 3 and up. While Series 1 and 2 watches will be supported by watchOS 6, they will have to wait for an update “later this year.”

Meanwhile, all we know about the Catalina release date is “October 2019.”

As the features and products become more and more intertwined and dependent on hardware, software, and cloud services, getting everything ready for release at the same time seems like an enormous, self-imposed burden.

Take, for example, the new Reminders application. All the new features in iOS 13 and Catalina means that the data is stored in a new format, which older versions of the Reminders application cannot read. Because of this, when you update your iPhone to the iOS 13, it will not synchronize with the Reminders app on the iPad or Mac, until you update those, which you cannot for several days or weeks.

Hypothetically, if Apple were to separate the Reminders app from the iOS release cycle, they could have waited for the update until all platforms, including iCloud, can support it. Users could then choose to update the app on all their platforms when it is convenient for them, without interrupting the user’s workflow.

Big feature laden software releases are a relic of the past when software was purchased on physical media in boxes. The big feature reveals at WWDC and then later at the release keynotes bring Apple a lot of attention and press. But, the chaos and delays (perceived and real) of the self-imposed release dates are not helping Apple’s reputation.

And yes, it is easy to criticize from the comfort of sitting behind my MacBook keyboard, while the amazing engineers and managers at Apple have to do all the work. There are probably people at Apple working this out in some way or another. Apple has put features into the interim “Spring” updates for High Sierra and Mojave and are deferring several features shown at WWDC to later versions of iOS 13 and Catalina.

How this continues to develop in the future will certainly be interesting.

If you would rather get the weekly newsletter by email, you can subscribe to the Scripting OS X Weekly Newsletter here!! (Same content, delivered to your Inbox once a week.)

#! On Scripting OS X

📰News and Opinion

🐦MacAdmins on Twitter

  • Eric Holtam: “DEP admins: “On September 19, Apple will release updated software license agreements. On that date a Program Administrator must go to the Apple Business Manager website and accept the following agreements to continue to use the program””
  • Scott Knight: “Need to do some quick hex editing? Just use VIM. Open your file and then :%!xxd to view the hex. Edit away and then %!xxd -r to switch back to the binary view. Save just like you would any other file in VIM.”
  • Arek Dreyer: “Apple Business Manager and Apple School Manager now available in China, Thailand, Saudia Arabia, and Vietnam. Wow!”
  • Rich Trouton: “Wow! Thanks to everyone who’s pre-ordered, you’ve pushed @cedge318 and my Apple device management book to the top of the charts on Amazon!” ( Pre-order on Amazon US, Amazon UK, Amazon DE, Affiliate Links)
  • Rosyna Keller: “The altool docs have been updated to include the new --store-password-in-keychain-item option, which allows you to create a keychain item specifically for use in notarization scripts that use -p "@keychain:<item name>"
  • Arek Dreyer: “Another reminder for the appropriate person in your organization to log in to Apple School Manager or Apple Business Manager and agree to the new terms and conditions.”

macOS 10.15 Catalina and iOS 13

🐞Bugs and Security

🔨Support and HowTos

🤖Scripting and Automation

🍏Apple Support

iOS 13

watchOS 6

“watchOS 6 is currently compatible with Apple Watch Series 3 and later. Apple Watch Series 1 and 2 will support watchOS 6 later this year.”

♻️Updates and Releases

🎧To Listen

📚 Support

There are no ads on my webpage or this newsletter. If you are enjoying what you are reading here, please spread the word and recommend it to another Mac Admin!

If you want to support me and this website even further, then consider buying one (or all) of my books. It’s like a subscription fee, but you also get a useful book or two extra!

Make zsh show working directory in Terminal window title in macOS

This subject has grown into a book: reworked and expanded with more detail and topics. Like my other books, I plan to update and add to it after release as well, keeping it relevant and useful. You can order it on the Apple Books Store now.

While I was working on customizing my zsh configuration files for my zsh article series, I noticed that Terminal would not display the current working directory when using zsh. it worked fine when I switched back to bash.

Note: showing the working directory in the window or tab title is enabled by default in Terminal. You can configure what Terminal shows in the title in the Preferences Window for each profile. Select a profile and

Some sleuthing showed me that /etc/bashrc_Apple_Terminal sets up a update_terminal_cwd function which sends some escape codes to Terminal to keep the window title bar updated. This file is sourced by /etc/bashrc for the Terminal.app.

In macOS 10.14 Mojave and earlier, this configuration file has no equivalent for zsh. /etc/zshrc has code that would load /etc/zshrc_Apple_Terminal, if it existed. However, this file does not exist in macOS Mojave and earlier.

It does in Catalina, though, and it sets up a similar function that is added into the precmd hook. If you have access to the Catalina beta, you can just copy the /etc/zshrc_Apple_Terminal to your Mojave (or earlier) Mac and it will work.

Alternatively, you can write your own implementation, which is what I did, because I wanted it before the Catalina files existed. My solution exists of the function and its setup in its own file. This file is in a directory that is added to the fpath so I can simply load it with autoload.

You can see the code for the function file here.

Then, in my .zshrc I load the function file with:

# path to my zsh functions folder:
my_zsh_functions=$repo_dir/dotfiles/zshfunctions/

# include my zshfunctions dir in fpath:
if [[ -d $my_zsh_functions ]]; then
    fpath=( $my_zsh_functions $fpath )
fi

# only for Mojave and earlier
if [[ $(sw_vers -buildVersion) < "19" ]]; then 
    # this sets up the connection with the Apple Terminal Title Bar
    autoload -U update_terminal_pwd && update_terminal_pwd
fi

And from now, the title will be updated in Mojave (and earlier) and the function won’t even be loaded in Catalina.

This might seem a bit overkill, now that the functionality is in the Catalina file, but it might serve as a useful example when you want to customize, or re-implement similar behavior.

Weekly News Summary for Admins — 2019-09-13

New iPhones! New Apple Watch! Even a new entry level iPad! The Apple iPhone event delivered mostly within expectations. I am not going to link all the detailed posts, you can go to Apple’s Newsroom for the official details.

We got release dates for iOS 13 (Sep 19), iOS 13.1 (Sep 30), iPadOS (Sep 30, unclear if this is iPadOS 13.0 or 13.1, but likely the latter) and macOS 10.15 Catalina (a vague ‘October’). (Check this MacStories post with all the dates, including hardware and services.)

So, there are a few extra weeks this year to test and prepare for the next macOS upgrade. It’ll be a busy fall either way. Make the most of it: Test, prepare, file bugs and feedback!

If you would rather get the weekly newsletter by email, you can subscribe to the Scripting OS X Weekly Newsletter here!! (Same content, delivered to your Inbox once a week.)

On Scripting OS X

News and Opinion

macOS 10.15 Catalina and iOS 13

MacAdmins on Twitter

  • Eric Holtam: “Notice: Do not try and restore BridgeOS on a machine running a beta OS with Configurator 2. Configurator doesn’t allow backward updates to previous BridgeOS versions and ”bricks“ the machine. ‘AMRestoreErrorDomain error 79 – backwards update not allowed: 17P50541b -> 16PXXX)’ ”
  • William Smith: “UltraThin Updates take 2: For tomorrow’s Microsoft Office for Mac update, Microsoft AutoUpdate will use UltraThin updates to move 16.28 > 16.29. Should be around 176 MB to update all of Office.”

Bugs and Security

Support and HowTos

Updates and Releases

To Watch

To Listen

Support

There are no ads on my webpage or this newsletter. If you are enjoying what you are reading here, please spread the word and recommend it to another Mac Admin!

If you want to support me and this website even further, then consider buying one (or all) of my books. It’s like a subscription fee, but you also get a useful book or two extra!

Notarize a Command Line Tool

There is an updated version of this post for the new tools in Xcode 13.

The upcoming macOS 10.15 Catalina will require more apps and tools to be notarized. Apple has somewhat loosened the requirements at last minute, but these changed limitations are only temporary, to give developers more time to adapt.

Notarizing Mac Application bundles has its pitfalls, but is overall fairly well documented. However, I have been working on some command line tools written in Swift 5 and figured out how to get those properly signed and notarized.

Howard Oakley has written up his experiences and that post was extremely helpful. But there were a few omissions and some steps that aren’t really necessary, so I decided to make my own write-up.

And yes, there is a script at the end…

Note: these instructions are for macOS 10.14.6 Mojave, Xcode 10.3 and Swift 5.0. It is very likely that the details will change over time.

Update 2019-09-24: Tested with Xcode 11 and it still works (the screen layout has changed for some of the options)

What do you need?

  • Apple Developer Account (Personal or Enterprise, the free account does not provide the right certificates)
  • Xcode 10.3 or 11
  • Developer ID Certificates (Application and Install)
  • Application Specific Password for your Developer account
  • a Command Line Tool Project that you want to sign and notarize

That’s a longish list. If you are already building command line tools in Xcode, you should have most of these covered already. We will walk through the list step-by-step:

Apple Developer Program Account

You need either the paid membership in the Apple Developer Program or be invited to an Apple Developer Enterprise Program team with access to the proper certificates.

You cannot get the required certificates with a free Apple Developer account, unless you are member of a team that provides access.

Xcode

You can download Xcode from the Mac App Store or the developer download page. When you launch Xcode for the first time, it will prompt for some extra installations. Those are necessary for everything to in the article to work.

Developer ID Certificates

There are multiple certificates you can get from the Developer Program. By default you get a ‘Mac Developer’ certificate, which you can use for building and testing your own app locally.

To distribute binaries (apps and command line tools) outside of the App Store, you need a ‘Developer ID Application’ certificate. To sign installer packages for distribution outside of the Mac App Store, you need a ‘Developer ID Installer’ certificate.

We will need both types of Developer ID certificates, the first to sign the command line tool and the second to sign and notarize the installer package.

If you have not created these yet, you can do so in Xcode or in the Developer Portal. If you already have the certificates but on a different Mac, you need to export them and re-import them on the new Mac. Creating new certificates might invalidate the existing certificates! So beware.

Once you have created or imported the certificates on your work machine, you can verify their presence in the Terminal with:

% security find-identity -p basic -v

This command will list all available certificates on this Mac. Check that you can see the ‘Developer ID Application’ and ‘Developer ID Installer’ certificates. If you are a member of multiple teams, you may see multiple certificates for each team.

You can later identify the certificates (or ‘identities’) by the long hex number or by the descriptive name, e.g. "Developer ID Installer: Armin Briegel (ABCD123456)"

The ten character code at the end of the name is your Developer Team ID. Make a note of it. If you are a member of multiple developer teams, you can have multiple Developer ID certificates and the team ID will help you distinguish them.

Application Specific Password for your Developer Account

Apple requires Developer Accounts to be protected with two-factor authentication. To allow automated workflows which require authentication, you can create application specific passwords.

Create a new application specific password in Apple ID portal for your developer account.

You will only be shown the password when you create it. Immediately create a ‘New Password Item’ in your Keychain with the following fields:

  • Keychain Item Name: Developer-altool
  • Account Name: your developer account email
  • Password: the application-specific password you just created

This will create a developer specific password item that we can access safely from the tools.

If you want, you can also store the app specific password in a different password manager, but the Xcode tools have a special option to use Keychain.

A Command Line Tool Project

You may already have a project to create a command line in Xcode. If you don’t have one, or just want a new one to experiment, you can just create a new project in Xcode and choose the ‘Command Line Tool’ template from ‘macOS’ section in the picker. The template creates a simple “Hello, world” tool, which you can use to test the notarization process.

My sample project for this article will be named “hello.”

Preparing the Xcode Project

The default settings in the ‘Command Line Tool’ project are suitable for building and testing the tool on your Mac, but need some changes to create a distributable tool.

Choosing the proper signing certificates

Before you can notarize the command line tool, it needs to be signed with the correct certificates.

  1. in Xcode, select the blue project icon in the left sidebar
  2. select the black “terminal” icon with your project’s name under the “Targets” list entry
  3. make sure the ‘General’ tab is selected
  4. under ‘Signing’ disable ‘Automatically manage signing’
  5. under ‘Signing (Debug)’ choose your Team and choose ‘Developer ID Application’ as the certificate
  6. under ‘Signing (Release)’ choose your Team and choose ‘Developer ID Application’ as the certificate
Setting the certificates
Setting the certificates

Enable Hardened Runtime

Enabling the ‘Hardened Runtime’ will compile the binary in a way that makes it harder for external process to inject code. This will be requirement for successful notarization starting January 2020.

  1. from the view where you changed the signing options, click on ‘Build Settings’ in the upper tab row
  2. click on ‘All’ to show all available settings
  3. enter ‘enable hardened’ in the search field, this will show the ‘Enable Hardened Runtime’ setting
  4. set the value in the project column (blue icon) to YES
Enable Hardened Runtime
Enable Hardened Runtime

Change the Install Build Location

If we want to automate the packaging and notarization, we need to know where Xcode builds the binary. The default location is in some /tmp subdirectory and not very convenient. We will change the location for the final binary (the ‘product’) to the build subdirectory in the project folder:

  1. in the same view as above, enter ‘Installation Build’ in the search field, this will show the ‘Installation Build Products Location’ setting
  2. double click on the value in the Project column (blue icon), this will open a popup window
  3. change the value to $SRCROOT/build/pkgroot
Change the Installation Build location
Change the Installation Build location

If you manage your code in git or another VCS, you want to add the build subdirectory to the ignored locations (.gitignore)

Build the Binary

You can use Xcode to write, test, and command line tool debug your. When you are ready to build and notarize a pkg installer, do the following:

  1. open Terminal and change directory to the project folder
  2. % xcodebuild clean install

This will spew a lot of information out to the command line. You will see a build subdirectory appear in the project folder, which will be filled with some directories with intermediate data.

After a successful build you should see a pkgroot directory in the build folder, which contains your binary in the usr/local/bin sub-path.

/usr/local/bin is the default location for command line tools in the Command Line Tool project template. It suits me fine most of the time, but you can change it by modifying the ‘Installation Directory’ build setting in Xcode and re-building from the command line.

Build the pkg

Command Line Tools can be signed, but not directly notarized. You can however notarize a zip, dmg, or pkg file containing a Command Line Tool. Also, it is much easier for users and administrators to install your tool when it comes in a proper installation package.

We can use the pkgroot directory as our payload to build the installer package:

% pkgbuild --root build/pkgroot \
           --identifier "com.example.hello" \
           --version "1.0" \
           --install-location "/" \
           --sign "Developer ID Installer: Armin Briegel (ABCD123456)" \
           build/hello-1.0.pkg

I have broken the command into multiple lines for clarity, you can enter the command in one line without the end-of-line backslashes \. You want to replace the values for the identifier, version and signing certificate with your data.

This will build an installer package which would install your binary on the target system. You should inspect the pkg file with Pacifist or Suspicious Package and do a test install on a test system to verify everything works.

If you want to learn more about installer packages and pkgbuild read my book “Packaging for Apple Administrators.”

Notarizing the Installer Package

Xcode has a command line tool altool which you can use to upload your tool for notarization:

xcrun altool --notarize-app \
             --primary-bundle-id "com.example.com" \
             --username "username@example.com" \
             --password "@keychain:Developer-altool" \
             --asc-provider "ABCD123456" \
             --file "build/hello-1.0.pkg"

The username is your developer account email.

The asc-provider is your ten digit Team ID. If you are only a member in a single team you do not need to provide this.

The password uses a special @keychain: keyword that tells altool to get the app-specific password out of a keychain item named Developer-altool. (Remember we created that earlier?)

This will take a while. When the command has successfully uploaded the pkg to Apple’s Notarization Servers, it will return a RequestUUID. Your notarization request will be queued and eventually processed. You can check the status of your request with:

xcrun altool --notarization-info "Your-Request-UUID" \
             --username "username@example.com" \                                    
             --password "@keychain:Developer-altool"   

Apple will also send an email to your developer account when the process is complete. I my experience this rarely takes more than a minute or two. (Being in Central EU time zone might be an advantage there). When the process is complete, you can run the above notarization-info command to get some details. The info will include a link that contains even more information, which can be useful when your request is rejected.

Note that the info links expire after 24 hours or so. You should copy down any information you want to keep longer.

Completing the Process

You will not receive anything back from Apple other than the confirmation or rejection of your request. When a Mac downloads your installer package and verifies its notarization status it will reach out to Apple’s Notarization servers and they will confirm or reject the status.

If the Mac is offline at this time, or behind a proxy or firewall that blocks access to the Apple Servers, then it cannot verify whether your pkg file is notarized.

You can, however, ‘staple’ the notarization ticket to the pkg file, so the clients do not need to connect to the servers:

% xcrun stapler staple build/hello-1.0.pkg

You can also use stapler to verify the process went well:

% xcrun stapler validate build/hello-1.0.pkg

But since stapler depends on the developer tools to be installed, you should generally prefer spctl to check notarization:

% spctl --assess -vvv --type install build/hello-1.0.pkg

Automating the Process

Obviously, I built a script to automate all this. Put the following script in the root of the project folder, modify the variables at the start of the script (lines 20–38) with your information, and run it.

The script will build the tool, create a signed pkg, upload it for notarization, wait for the result, and then staple the pkg.

You can use this script as an external build tool target in Xcode. There are other ways to integrate scripts for automation in Xcode, but all of this is a new area for me and I am unsure which option is the best, and which I should recommend.

Links and Videos

These links and videos, especially Howard Oakley’s post and Tom Bridge’s PSU Presentation have been hugely helpful. Also thanks to co-worker Arnold for showing me this was even possible.

Going forward

Notarization is a key part of Apple’s security strategy going in macOS.

As MacAdmins we will usually deploy software through management systems, where the Gatekeeper mechanisms which evaluate notarization are bypassed. There are, however, already special cases (Kernel Extensions) where notarization is mandatory. It is likely that Apple will continue to tighten these requirements in the future. The macOS Mojave 10.14.5 update has shown that Apple may not even wait for major releases to increase the requirements.

If you are building your own tools and software for macOS and plan to distribute the software to other computers, you should start signing and notarizing.

On the other hand, I find the introduction of Notarization to macOS encouraging. If Apple wanted to turn macOS into a “App Store only system” like iOS, they would not have needed to build the notarization process and infrastructure. Instead, Apple seems to have embraced third-party-software from outside the App Store.

Notarization allows Apple to provide a security mechanism for software distributed through other means. It cannot be 100% effective, but when used correctly by Apple and the software developers it will provide a level of validation and trust for software downloaded from the internet.

Weekly News Summary for Admins — 2019-09-06

You can tell from the various articles and support posts that release day is definitely getting closer.

iOS 10.13.1 beta 2 was release and it is questionable if we will see another beta of either iOS, iPadOS or macOS before release. Apple has loosened notarization requirements for macOS Catalina temporarily (until Jan 2020) to help developer handle the transition.

If you would rather get the weekly newsletter by email, you can subscribe to the Scripting OS X Weekly Newsletter here!! (Same content, delivered to your Inbox once a week.)

Headlines

  • Rich Trouton’s and Charles Edge’s “Apple Device Management: A Unified Theory of Managing Macs, iPads, iPhones, and AppleTVs” book is available for pre-order on Amazon US, UK, DE (Affiliate Links)

On Scripting OS X

News and Opinion

macOS Catalina and iOS 13

MacAdmins on Twitter

  • Mike Boylan:
    “This is true. Age != skill (in both directions). Am fortunate that most people have treated me with respect.” (Thread)
  • Jason Fried: “When Google puts 4 paid ads ahead of the first organic result for your own brand name, you’re forced to pay up if you want to be found. It’s a shakedown. It’s ransom. But at least we can have fun with it. Search for Basecamp and you may see this attached ad.” (Image)
  • Benedict Evans: “Hell just froze over. beta.music.apple.com
  • TimPerfitt: ‘Catalina “Restore from snapshot”’ (Image)

Bugs and Security

Support and HowTos

Apple Support

Updates and Releases

To Watch

To Listen

Just for Fun

Support

There are no ads on my webpage or this newsletter. If you are enjoying what you are reading here, please spread the word and recommend it to another Mac Admin!

If you want to support me and this website even further, then consider buying one (or all) of my books. It’s like a subscription fee, but you also get a useful book or two extra!

Get Current User in Shell Scripts on macOS

…or, how to deal with deprecated bash and python…

There are many solutions to get the current logged in user in macOS to use in a shell script. However, the semi-official, “sanctioned” method has always involved a rather elaborate python one-liner, originally published by Ben Toms:

loggedInUser=$(/usr/bin/python -c 'from SystemConfiguration import SCDynamicStoreCopyConsoleUser; import sys; username = (SCDynamicStoreCopyConsoleUser(None, None, None) or [None])[0]; username = [username,""][username in [u"loginwindow", None, u""]]; sys.stdout.write(username + "\n");')

Update, because this a FAQ:

There are various other solutions to get the current user which use stat, who, or last commands. These will work in most situations. But there are edge cases, mostly with Fast User Switching, where these methods don’t return the correct user.

From python to scutil

While this has worked wonderfully and more reliably than other solutions, it always looked inelegant, and added a dependency on the python binary. This used to be fine, as the python binary shipped with macOS. But in the macOS Catalina release notes, we have learned that some, yet undefined, future version of macOS will not include the Python, Ruby, and Perl interpreters any more.

With prescient timing, Erik Berglund figured out a different method, which still accesses the same system framework, but through the scutil command rather than the PyObjC bridge.
Over time, different contributors in the MacAdmins Slack have optimized this command:

loggedInUser=$( scutil <<< "show State:/Users/ConsoleUser" | awk '/Name :/ && ! /loginwindow/ { print $3 }' )

Sidenote: the scutil version runs 15–20x faster than the python version. Speed is not usually a major consideration for MacAdmin scripts, but it’s a nice improvement.

From bash to sh

This works great in bash and zsh. However, the built-in bash has been deprecated as well. (I have talked about this, a lot.)

While zsh is a great replacement for bash for interactive terminal use and scripting on the ‘full’ macOS system, there are environments (Recovery) where zsh is not present.

The only shell Mac Admins can rely on in all contexts is now /bin/sh.

Because of this I recommend using /bin/sh for installer scripts (pre- and postinstall scripts in pkgs).

The Posix sh standard does not include the ‘here string’ <<< used in Erik’s command. Nevertheless, when you use the above construct in sh on macOS it will still work fine. This is because sh on macOS is actually handled by bash in sh compatibility mode.

And while bash in sh compatibility mode will recreate the odd syntax and quirks of sh for compatibility, it will also happily run ‘bashims’ that don’t even exist in sh. E.g. double brackets [[...]] or here strings <<<.

As long as bash is taking care of sh scripts, you will be fine. Myself, I would not have noticed this if I had not ran a sh script through shellcheck. Shellcheck told me that Posix sh should not understand the here string:

loggedInUser=$( scutil <<< "show State:/Users/ConsoleUser" | awk '/Name :/ && ! /loginwindow/ { print $3 }' )
                       ^-^ SC2039: In POSIX sh, here-strings are undefined.

But we’re fine on macOS, because sh is really bash right? No need to worry…

Predicting the demise of bash

We don’t yet know when, but the /bin/bash binary will eventually be removed from macOS. The version included with macOS has not been updated since 2014. It is unlikely that the twelve year old bash v3.2 will receive patches for security vulnerabilities in the future.

With the switch to zsh as the default shell in Catalina Apple is laying the groundwork to remove bash v3.2 from the system in a future release. cron and login hooks were infamously deprecated in macOS years ago and are still around, so we could be looking at years. But my suspicion is that bash is just one security vulnerability away from being removed.

With the High Sierra and Mojave updates, Apple has shown that they do not need to wait for major releases to remove or add functionality or restrictions to the system.

I believe /bin/bash has more than a few months still. However, Apple’s messaging on this switch is uncharacteristically strong. It would not be terribly surprising if the Catalina Spring Update in March 2020 removed bash.

The amount of old installer packages which would break on such a change is terrifying. Apple might be willing to pay this price, to avoid a publicized security vulnerability. A severe bash vulnerability would definitely get a lot of publicity. Remember Shellshock? That was the last time bash v3.2 was patched.

Instead of removing /bin/bash Apple might be able to replace it with zsh in bash emulation mode. So far, I have found no signs for this. While it wouldn’t be perfect, it would be a safer move.

Python 2.7 will still get updates until the end of 2019. There are also some commands and tools in the system written in python 2.7. It will be harder for Apple to migrate these to other languages, so I think Python has a longer countdown clock. But it cannot hurt to start preparing now.

Whither sh?

Whenever /bin/bash is removed, what will happen with /bin/sh at that point? The answer can be deduced from the support article on the shell switch.

How to test your shell scripts

To test script compatibility with Bourne-compatible shells in macOS Catalina, you can change /var/select/sh to /bin/bash, /bin/dash, or /bin/zsh. If you change /var/select/sh to a shell other than bash, be aware that scripts that make use of bashisms may not work properly.

zsh can be made to emulate sh by executing the command zsh --emulate sh.

We can actually see that Apple has two options here. sh processes could be run by zsh in emulation mode or by dash.

dash is the Debian Almquist shell, which is a minimal implementation of the Posix sh standard. dash is a newcomer to macOS on Catalina. New arrivals on a release where Apple seems intent on cleaning out ballast (32-bit, Kernel Extensions, bash v3, Python, Perl, and Ruby) are immediately of interest.

When you check on Catalina Recovery system, you can see that /bin/dash has been added, but /bin/zsh is absent. (/bin/bash is still present on Catalina Recovery and the sh binary is also still bash.)

Putting all of this together, I would hazard that Apple is planning to use dash to take over from ‘bash as sh.’ This will also bring macOS in line with most other Unix and Linux distributions where dash commonly handles sh processes.

With the symbolic link mechanism described in the support article, Apple could switch sh to dash in an update while bash is still present. This could allow Developers or Mac Admins to switch their managed systems back to bash when they need to mitigate problems.

When Apple switches to dash, sh scripts that use bashisms will break.

You can start testing this in Catalina by changing the symbolic link at /var/select/sh:

% sudo ln -sf /bin/dash /var/select/sh

(Change /bin/dash to /bin/bash to revert to default.)

When I test the above command to get the current user with dash, it returns:

Syntax error: redirection unexpected

(You can also change the shebang of a script you want to test to #!/bin/dash then you do not have to change the system sh.)

Getting the Current User in sh, Future Proof (I hope)

The solution is easy enough. Replace the here string with a pipe. For sh scripts, change the syntax to:

loggedInUser=$( echo "show State:/Users/ConsoleUser" | scutil | awk '/Name :/ && ! /loginwindow/ { print $3 }' )

This will make Posix sh, dash, and shellcheck happy and will still work in bash and zsh.

This syntax (echoing into a pipe) is considered inelegant and inefficient, because it requires an additional process (echo) and an additional pipe. Here docs and here strings were introduced exactly to avoid this. Obviously, if you prefer, you can continue to use the here string form of the command for bash and zsh.

Conclusion

There is no need panic and search and replace the python command in all your scripts immediately. I would recommend replacing this line when you update a script. Keep all of this in mind when working on your administration, installation and workflow scripts going forward.

You may also find scripts in third party documentation and installers that need updating, and you should let the author or vendor know.

  • /bin/bash (v3.2) is deprecated and has not received updates since 2014
  • /usr/bin/python (v2.7) is deprecated and will not receive new updates starting in Jan 2020
  • both will eventually be removed from macOS
  • we don’t know when exactly, could be years, could be months
  • even when you do not use Python for scripting, you may be using Python ‘one-liners’ in shell scripts
  • bash scripts should be moved to zsh (general use) or sh (general and installation scripts)
  • sh processes in macOS will likely be handled by dash in the future
  • use shellcheck to find and correct bashisms in sh scripts
  • shellcheck does not work for zsh scripts

Weekly News Summary for Admins — 2019-08 -30

Things are heating up for September release season. We got new betas for iOS and iPadOS 13.0 and macOS Catalina beta7. Surprisingly, there also is a beta for iOS 13.1, which includes some features shown at WWDC, which were removed from the 13.0 beta. While I generally approve of this flexibility, it makes testing confusing and more complex.

We also got a second supplemental update for 10.14.6, fixing some more bugs and security holes. The newest 10.14.6 has the build number 18G95. The silver lining in all this version chaos is, that we will likely not end Mojave on a forked hardware build.

Apple sent out the invitations to the iPhone event on September 10, 10am PDT (19:00 CEST). If previous years can be used as a guide, this could mean the iOS 13.0 release will be on September 13 or 20 and the macOS 10.15.0 Catalina release on September 20 or 27.

If you would rather get the weekly newsletter by email, you can subscribe to the Scripting OS X Weekly Newsletter here!! (Same content, delivered to your Inbox once a week.)

On Scripting OS X

The “Moving to zsh” class on September 6 is now full. Looking forward to seeing you all there! If you missed to sign up for this class, the next “Scripting macOS” beginners’ class on October 30–31 will be updated for Catalina and contain plenty of zsh, as well.

News and Opinion

macOS 10.13 Catalina and iOS 13

Bugs and Security

Support and HowTos

Scripting and Automation

Apple Support

Updates and Releases

To Listen

Support

There are no ads on my webpage or this newsletter. If you are enjoying what you are reading here, please spread the word and recommend it to another Mac Admin!

If you want to support me and this website even further, then consider buying one (or all) of my books. It’s like a subscription fee, but you also get a useful book or two extra!

Check Installer Pkgs for deprecated scripts

macOS 10.15 Catalina will deprecate the built-in /bin/bash. I have talked about this at length.

The release notes for Catalina also tell us that other built-in scripting runtimes, namely Python, Perl, and Ruby. Will not be included in future macOS releases (post-Catalina) any more.

This means, that if you want to use bash, Python, Perl, or Ruby on macOS, you will have to install, and maintain your own version in the future.

However, scripts in installation packages, cannot rely on any of these interpreters being available in future, post-Catalina versions of macOS. Installer pkgs can be run in all kinds of environments and at all times, and you would not want them to fail, because a dependency is missing.

The good news is that we still have time. All the runtimes mentioned above are still present in Catalina, so the packages will continue to work for now. But if you are building installation scripts, you need to check if any of the installation scripts use one of these interpreters and fix them.

I recommend to use /bin/sh for installation scripts, since that will run in any macOS context, even the Recovery system.

If you are using third-party installer packages, you may also want to check them for these interpreters, and notify the developer that these packages will break in future versions of macOS.

To check a flat installer package, you would expand it with pkgutil --expand and then look at script files in the Scripts folder. This will work fine for a package or two, but gets tedious really quickly, especially with large distribution pkgs with many components (e.g. Office).

So… I wrote a script to do it. The script should handle normal component pkgs, distribution pkgs and the legacy bundle pkgs and mpkgs.

You can get the pkgcheck script from my Github repo.

What the script does

Once I had written the code to inspect all these types of pkgs, I realized I could grab all other kinds of information, as well. The pkgcheck.sh script will check for:

  • Signature and Notarization
  • Type of Package: Component, Distribution, legacy bundle or mpkg
  • Identifier and version (when present)
  • Install-location
  • for Distribution and mpkg types, shows the information for all components as well
  • for every script in a pkg or component, checks the first line of the script for shebangs of the deprecated interpreters (/bin/bash, /usr/bin/python, /usr/bin/perl, and /usr/bin/ruby) and print a warning when found

How to run pkgcheck.sh

Run the script with the target pkg file as an argument:

% ./pkgcheck.sh sample.pkg

You can give more than one file:

% ./pkgcheck.sh file1.pkg file2.pkg ...

When you pass a directory, pkgcheck.sh will recursively search for all files or bundle directories with the pkg or mpkg extension in that directory:

% ./pkgcheck.sh SamplePkgs

Features and Errors

There are a few more things that I think might be useful to check in this script. Most of all, I want to add an indicator whether a component is enabled by default or not. If you can think of any other valuable data to display, let me know. (Issue or Pull Request or just ping me on MacAdmins Slack)

I have tested the script against many pkgs that I came across. However, there are likely edge cases that I haven’t anticipated, which might break the script. If you run into any of those, let me know. (File an Issue or Pull Request.) Having the troublesome pkg would of course be a great help.

Note: the script will create a scratch directory for temporary file extractions. The script doesn’t actually expand the entire pkg file, only the Scripts sub-archive. The scratch folder will be cleaned out at the beginning of the next run, but not when the script ends, as you might want to do some further inspections.

Sample outputs

This is a sample pkg I build in my book, it has pre- and postinstall scripts using a /bin/bash shebang:

% ./pkgcheck.sh SourceCodePro-2.030d.pkg
SourceCodePro-2.030d
SamplePkgs/SourceCodePro-2.030d.pkg
Signature:      None
Notarized:      No
Type:           Flat Component PKG
Identifier:     com.example.SourceCodePro
Version:        2.030d
Location:       /
Contains 2 resource files
postinstall has shebang #!/bin/bash
preinstall has shebang #!/bin/bash

This is the experimental notarized pkg installer for desktoppr:

% ./pkgcheck.sh desktoppr-0.2.pkg
desktoppr-0.2
SamplePkgs/desktoppr-0.2.pkg
Signature:      Developer ID Installer: Armin Briegel (JME5BW3F3R)
Notarized:      Yes
Type:           Flat Component PKG
Identifier:     com.scriptingosx.desktoppr
Version:        0.2
Contains 0 resource files

And finally, this is a big one, the Microsoft Office installer: (they have some work to do to clean up those scripts)

% ./pkgcheck.sh Microsoft\ Office\ 16.27.19071500_Installer.pkg
Microsoft Office 16.27.19071500_Installer
SamplePkgs/Microsoft Office 16.27.19071500_Installer.pkg
Signature:      Developer ID Installer: Microsoft Corporation (UBF8T346G9)
Notarized:      No
Type:           Flat Distribution PKG
Contains 11 component pkgs

    Microsoft_Word_Internal
    Type:           Flat Component PKG
    Identifier:     com.microsoft.package.Microsoft_Word.app
    Version:        16.27.19071500
    Location:       /Applications
    Contains 3 resource files

    Microsoft_Excel_Internal
    Type:           Flat Component PKG
    Identifier:     com.microsoft.package.Microsoft_Excel.app
    Version:        16.27.19071500
    Location:       /Applications
    Contains 2 resource files

    Microsoft_PowerPoint_Internal
    Type:           Flat Component PKG
    Identifier:     com.microsoft.package.Microsoft_PowerPoint.app
    Version:        16.27.19071500
    Location:       /Applications
    Contains 2 resource files

    Microsoft_OneNote_Internal
    Type:           Flat Component PKG
    Identifier:     com.microsoft.package.Microsoft_OneNote.app
    Version:        16.27.19071500
    Location:       /Applications
    Contains 2 resource files

    Microsoft_Outlook_Internal
    Type:           Flat Component PKG
    Identifier:     com.microsoft.package.Microsoft_Outlook.app
    Version:        16.27.19071500
    Location:       /Applications
    Contains 2 resource files

    OneDrive
    Type:           Flat Component PKG
    Identifier:     com.microsoft.OneDrive
    Version:        19.70.410
    Location:       /Applications
    Contains 30 resource files
    postinstall has shebang #!/bin/bash
    od_logging has shebang #!/bin/bash
    od_service has shebang #!/bin/bash
    od_migration has shebang #!/bin/bash
    preinstall has shebang #!/bin/bash

    Office16_all_autoupdate
    Type:           Flat Component PKG
    Identifier:     com.microsoft.package.Microsoft_AutoUpdate.app
    Version:        4.13.19071500
    Location:       /Library/Application Support/Microsoft/MAU2.0
    Contains 2 resource files
    postinstall has shebang #!/bin/bash
    preinstall has shebang #!/bin/bash

    Office16_all_licensing
    Type:           Flat Component PKG
    Identifier:     com.microsoft.pkg.licensing
    Version:        16.27.19071500
    Location:       /
    Contains 2 resource files
    dockutil has shebang #!/usr/bin/python

    Office_fonts
    Type:           Flat Component PKG
    Identifier:     com.microsoft.package.DFonts
    Version:        0
    Location:       /private/tmp/com.microsoft.package.DFonts
    Contains 1 resource files
    postinstall has shebang #!/bin/bash

    Office_frameworks
    Type:           Flat Component PKG
    Identifier:     com.microsoft.package.Frameworks
    Version:        0
    Location:       /private/tmp/com.microsoft.package.Frameworks
    Contains 1 resource files
    postinstall has shebang #!/bin/bash

    Office_proofing
    Type:           Flat Component PKG
    Identifier:     com.microsoft.package.Proofing_Tools
    Version:        0
    Location:       /private/tmp/com.microsoft.package.Proofing_Tools
    Contains 1 resource files
    postinstall has shebang #!/bin/bash

Weekly News Summary for Admins — 2019-08-23

Summer vacation time is coming to an end and the beta season for macOS Catalina and iOS 13 is heating up! Another Catalina beta dropped this week, and third parties are now starting to announce their own betas and/or preliminary support.

If you would rather get the weekly newsletter by email, you can subscribe to the Scripting OS X Weekly Newsletter here!! (Same content, delivered to your Inbox once a week.)

On Scripting OS X

The “Moving to zsh” class on September 6 is now full. Looking forward to seeing you all there! If you missed to sign up for this class, the next “Scripting macOS” beginners’ class on October 30–31 will be updated for Catalina and contain plenty of zsh, as well.

News and Opinion

macOS 10.15 Catalina

MacAdmins on Twitter

  • William Smith: “Excited for this release from Microsoft because it’ll be the first app they’ve ever released for Mac that’s fully manageable from day 1. There’s been a lot of outreach the past few years from them to MacAdmins to make something like this possible.”
  • Mike Stern: “SF Mono is now available! Also, SF Pro and SF Compact have been updated to reflect some additions and changes to SF Symbols. developer.apple.com/fonts

Bugs and Security

Support and HowTos

Scripting and Automation

Apple Support

Updates and Releases

To Listen

Just for Fun

Support

There are no ads on my webpage or this newsletter. If you are enjoying what you are reading here, please spread the word and recommend it to another Mac Admin!

If you want to support me and this website even further, then consider buying one (or all) of my books. It’s like a subscription fee, but you also get a useful book or two extra!