Launching Scripts #1: From Terminal

Scripts, no matter which language they are written in, are an incredibly important part of a MacAdmin’s tool kit.

Most posts, books and tutorials focus on how to write scripts. Admittedly, the skill to make the computer and all its software running do what you want is very important. But once you have a working script, you will have to consider how to launch the script at the right time and with the right input.

When I started to think about this, I realized macOS provides many ways to launch or schedule scripts. Some are fairly obvious, others are quite obscure and maybe even frivolous.

Over the next few weeks (i.e. when I find time), I will publish a series of posts describing various ways of launching scripts and workflows on macOS. This post starts with the most obvious: launch a script from Terminal.

The Basics: Launch from Terminal

It might seem trivially obvious that you can launch scripts and workflows from Terminal. Nevertheless, it does require some explanation and context. Launching from Terminal is the most basic form of launching a script. As such, it defines the ‘baseline’ experience of how we expect scripts to run.

Command lookup

When you enter something into a Terminal prompt and hit the return key, the shell will split the text on whitespace. The first element of the text is the command. If that text contains a / character, the shell will interpret the entire element as a path to the executable. The path can be relative or absolute.

If the first element does not contain a / then the shell will first look for functions, aliases and built-in commands, in that order. Then the shell will search for a command executable using the colon-separated paths in the PATH environment variable. It will use the first executable it finds.

If the shell fails to find something to execute, it will show an error.

This is why you need to the ./ to execute scripts in the current working directory. Even though the ./ seems redundant, it tells the shell to look for the executable in the current working directory, instead of using the PATH.

You can find this process described in more detail here. I also have a post on how to configure your PATH variable.

Shebang

Scripts are ‘just’ text files. Two things distinguish them from normal text files.

First, the executable bit is set to tell the system this file contains executable code. You do this with the chmod +x my_script.sh command. Here comes the trick, though. The text in the script file isn’t really executable code that the CPU can understand directly. It needs an ‘interpreter’ which is another program that, wells, interprets the text file and can tell the CPU what to do.

The interpreter for a script file is set in the first line of the script with the #! character combination. The #! is also called ‘hashbang’ or ‘shebang.’ The shebang is followed by an absolute path to the interpreter, e.g. /bin/sh, /bin/bash, or /usr/local/bin/python.

Note: the shebang has to be in the very first line (actually the first characters) of the script, you cannot have comments (or anything else) before it.

Note: you often see shebangs using the env command in the form #!/usr/bin/env bash. These are useful for cross-platform portability, but have trade-offs. I have an article just for that.

Arguments

The shell is only interested in the first part of the command line text you entered. The entire list of parts (or elements, split on whitespace) are passed into the command as its arguments. A shell script sees the first element, the command or path to the command as $0 and the the remaining arguments as $1, $2, and so on.

Other languages will handle this in a similar fashion, arrays or lists of arguments are common.

Environment

Shells also have environment variables. We already talked about the role of the PATH variable in the command lookup. In Terminal, you can list all variables in your current environment with the env command. Some of these environment variables can be extremely useful, like SHELL, USER, and HOME.

When you launch a script from Terminal, a new shell environment is created and all of the environment variables are copied. This is great, beacuse the script can access the environment variables, and might even change them. But the changes in the script’s sub-shell, will not affect the current shell. In other words, a script can change the PATH environment variable in its own context, without changing yours.

In other Unix-based systems, environment variables are a common means of providing data to apps and processes. In the interactive shell on macOS, environment variables are very useful for this purpose. We will see, however, that when you launch processes and scripts through other means, environment variables can be a challenge on macOS.

Note: A shell environment also contains shell options, which are also inherited to sub shells.

Input and Output

Shell scripts and tools also have input and output. When you launch a script from the the Terminal, both output streams (Standard Output and Standard Error) are connected to the Terminal, so the output will be shown in the Terminal window.

When you use a script with pipes, then its Standard Input (stdin) will be connected to the previous tool’s Standard Out (stdout). The last script’s stdout and stderr will be shown in Terminal.

One thing that is special about interactive Terminal input and output, is that it happens while the script or tool is running. That means you can get live updates on the progress.

When we launch scripts in other contexts, their input and output may be buffered. This means that the system waits for the script or tool process to complete before piping its output to the next tool or to the process that called it. This is not something you usually have to worry about, but again it is something you should have in mind when running scripts in non-Terminal contexts.

Conclusion

While glaringly obvious, launching a script from Terminal does have some intricacies. This post sets a ‘baseline’ for how scripts work on macOS. In future installments, we will re-visit some of these topics, and the differences will become relevant when we launch scripts in contexts other than an interactive shell.

In the next post, learn how to create a double-clickable file to launch a script from Finder.

3 thoughts on “Launching Scripts #1: From Terminal”

  1. Regarding shebang, a more system independent way would be to use e.g. ‘#!/usr/bin/env bash’. With this the script would also be usable on FreeBSD, where bash would be installed from FreeBSD Ports and then be available as ‘/usr/local/bin/bash’ or some Linux systems (e.g. Debian) where it is ‘/usr/bin/bash’.
    Unfortunately there are some other unixode systems around, where ‘env’ is not in ‘/usr/bin/’ and so the shebang needs to be adjusted.

    1. You make an excellent point. But the env shebang doesn’t just magically add portability. It has trade-offs. Your note on how the env binary may not be in /usr/bin on all platforms, scratches at these trade-offs, but there are more. Let me elaborate:

      Using a /usr/bin/env shebang can get you some level of portability. However, with this also comes with a loss of predictability and reliability.

      When used in the shebang, the env binary will use the current PATH to lookup the interpreter binary for the script.

      Imagine you have installed bash v5 on your Mac. This will (usually) put a bash 5 binary at /usr/local/bin/bash. When you run a script from the interactive shell, the env will find the bash v5 binary first, so your script will be interpreted with bash v5. This is probably what you want, since you installed bash v5 in the first place.

      When you run the same script on a different Mac (same macOS version, but it doesn’t have bash v5 installed) env will pick up /bin/bash. Your script will work even though that other Mac doesn’t have /usr/local/bin/bash, so you gained portability.

      However, /bin/bash is bash v3.2, so your script might behave differently (i.e. fail if it uses bash 5 features), so you lost either predictability and reliability, or you lose (bash v5) functionality. To retain reliability, you can restrict the script to features that work in both bash versions, but then env gives you no advantage, and you might as well use /bin/bash as the shebang.

      Even on the same Mac the PATH variable may be different depending on the context. For example, when you run your script with the AppleScript do shell script command (a later post in this series), the PATH in that context is not the same as the PATH in your interactive shell. Most importantly, it does not contain /usr/local/bin, so env will not ‘see’ the bash 5 binary there. The same script with the same user on the same computer, will behave differently when run in a different context.

      It gets worse with python and python3 and other run time interpreters, where it is likely that there are multiple different versions installed and the behavior between versions varies more. (My Mac currently has four different python 3 binaries, and I am not even a Python developer.) Also with python3 there is a risk a script might trigger the ‘you have to install Developer Command Line Tools’ dialog when Xcode is not installed. (I know developers have a hard time considering that there are Macs without Xcode installed.)

      (With the demise of python 2 in macOS 12.3, I saw some react by changing the shebang in their python scripts from /usr/bin/python to /usr/bin/env python which solves nothing. Some switched to /usr/bin/env python3 which often makes things worse.)

      As a MacAdmin, my scripts don’t need to be portable to other systems. They use tools and access files and folders that only exist on macOS. Predictability and reliability, however, are paramount. Configuration and installation scripts need to run reliably on thousands of Macs regardless of what else is installed. In this context, there is no gain in using /usr/bin/env over an absolute shebang.

      That said, using a /usr/bin/env shebang can be the best solution for workflows where cross-platform portability is of value. But he portability does not come magically from just switching the shebang to /usr/bin/env. To get reliable behavior, the systems you are porting between need to be well managed, with a predictable setup and configuration of the interpreter binary and environment.

      When your scripts work across systems with the env shebang, it is thanks to the efforts of the platform developers and your sys admins/devops for creating and maintaining this consistency.

      Alternatively, you keep the scripts so simple, that the differences between interpreter versions and environments have no impact.

      Using POSIX sh, instead of bash or zsh, is another option to gain portability, but that has its own trade-offs, mostly in functionality.

      Thank you for the comment. I realize this is important. I will also add this reply to the post itself.

Leave a Reply

Your email address will not be published.

This site uses Akismet to reduce spam. Learn how your comment data is processed.