There was a comment to my previous post about using the
/usr/bin/env shebang to gain system portability.
Regarding shebang, a more system independent way would be to use e.g. ‘#!/usr/bin/env bash’. With this the script would also be usable on FreeBSD, where bash would be installed from FreeBSD Ports and then be available as ‘/usr/local/bin/bash’ or some Linux systems (e.g. Debian) where it is ‘/usr/bin/bash’. Unfortunately there are some other unixode systems around, where ‘env’ is not in ‘/usr/bin/’ and so the shebang needs to be adjusted.
(I replied to the comment there, but then realized this deserves its own post.)
/usr/bin/env shebang provides a means for system portability. It has many valid use cases. However, you don’t just magically gain portability by switching to an
env shebang. There are many trade-offs to consider.
The note on how the
env binary may not be in
/usr/bin on all platforms, hints at some of these trade-offs, but there are more.
The trade-offs are a loss of predictability and reliability, or functionality, or increased maintenance and management.
Let me elaborate…
How the /usr/bin/env shebang works
When used in the shebang, the
/usr/bin/env binary will use the current environment’s
PATH to lookup the interpreter binary for the script, in the same way the shell looks up commands.
As an example, let us imagine you have installed bash v5 on your Mac. Either manually or using
brew or some other package management system.
This will (usually) put the bash v5 binary at
/usr/local/bin directory is a common choice for custom command line tools, because it is part of the default
PATH for interactive shells on macOS and not protected by SIP/SSV. The default
PATH on macOS for interactive shells is:
Some installations will put the binary in a different location in the file system. Then you you have to pre-pend the directory containing the binary to your
PATH variable in your shell configuration. The order of the directories in the
PATH is important, because the shell and
env will stop the search when they find the first match. If
/usr/local/bin came after
/bin in the
PATH the new binary would not be ‘seen’ since the pre-installed, old
/bin/bash binary is found first.
Some installations solve this by placing a symbolic link to the binary in
When you run a script from the interactive shell with a shebang of
#!/usr/bin/env bash, then
env would find the bash v5 binary first in
/usr/local/bin, so your script is interpreted with bash v5. This is probably what you were hoping for, when you installed bash v5 in the first place.
When you run the same script on a different Mac (same macOS version, but it doesn’t have bash v5 installed)
env will pick up
/bin/bash. Your script will work even though that other Mac doesn’t have
/usr/local/bin/bash, so you gained portability.
/bin/bash is bash v3.2, so your script may behave differently. If the script uses bash v5 features that are not available in the 15-year-old bash v3.2, it will generate errors. Since you actively chose to install bash v5 on the first Mac, it is likely you needed some of these bash v5 features, so it is likely your script will fail on other Macs, which don’t have bash v5 installed.
You lost either predictability and reliability (which version and features are available? Does my script run successfully?), or you lose functionality (the features added to bash v5 since v3.2). To retain reliability, you can restrict the script to features that work in both bash versions. But then using the
env shebang gives you no advantage, and you might as well use
/bin/bash as the shebang.
One alternative is to use a
/usr/local/bin/bash shebang for scripts which use bash v5 functionality and continue to use
/bin/bash for scripts that need to run across multiple Macs, where you pay attention to using only features available in bash v3.2. You gain predictability and reliability, but your bash v5 scripts aren’t portable to other Macs. They may even fail on other Macs with bash v5 installed, if the bash v5 binary is installed in a different location.
When you use
/usr/bin/env bash for a bash v5 script, it will run fine on all Macs which have bash v5 installed and the
PATH configured properly to find it. (Configuring and maintaining the
PATH does not happen on its own.) But the script will still fail on Macs without any bash v5. You can (and probably should) add a version check to the script, but now you are increasing code maintenance.
When you are managing a fleet of Macs, you also have the option (or in this case, I would say, the duty) to install bash v5 in a consistent location and version across all Macs in your fleet and pre-configure the proper
PATH in the contexts the script will run in. Then you get predictability and functionality, but it requires extra effort in deployment and maintenance.
This requires a decently experienced MacAdmin and the proper tooling, neither of which comes for free.
Note: There are great open source solutions for macOS in this area, but I consider them ‘free, as in puppy,’ so they come with higher skill requirements and/or maintenance effort for the admin. And this isn’t supposed to imply that all commercial solutions are ‘easy to use,’ either. It’s trade-offs all the way down.
Context changes the
Notice that so far I kept talking about “the default
PATH for the interactive shell.”
PATH variable may be different depending on the context, even on the same Mac with the same user. For example, when you run your script with the AppleScript
do shell script command, the
PATH in that context is not the same as the
PATH in your interactive shell. It will be:
You can verify this by opening Script Editor and running the
do shell script "echo $PATH". Other context, like scripts in installation packages will see other
Most importantly, the
PATH in these other contexts, does not contain
/usr/local/bin, or any other addition you made to your
PATH in the shell configuration files. An
/usr/bin/env shebang will not ‘see’ a bash 5 binary you installed on the system. The same script with the same user on the same computer, will behave differently when run in a different context.
PATH values are an intentional choice. In these contexts, especially installation package scripts, reliability and predictability are extremely important. You do not want user and third-party installed custom binaries to interfere with the command lookup.
Sidenote on Python
python3 and other run time interpreters, it gets even more difficult. There may multiple different versions installed and the behavior and functionality between versions varies more. My Mac currently has four different Python 3 binaries, each with a different version, and I not even remotely a full-time Python developer. When you call
python3 on a non-developer Mac it will trigger the ‘You have to install Developer Command Line Tools’ dialog when Xcode is not installed. (Developers seem to have a hard time considering that there are Macs without Xcode installed.)
With the demise of python 2 in macOS 12.3, some developers reacted by changing the shebang in their python scripts from
/usr/bin/env python which solves nothing, when the binary goes away without replacement. Some switched to
/usr/bin/env python3 which can makes things worse, by triggering the Developer Tools installation or picking a random python3 binary of the wrong version.
The only reliable solution for the mess that is
python is to deploy a consistent version of Python 3 in a consistent location. You can do this by either bundling the python framework and module your tool needs together with the tool, or by deploying and maintaining the Python frameworks and modules with a management system.
As a MacAdmin, my scripts don’t need to be portable to systems other than macOS. They usually use tools and access files and folders that only exist on macOS. Predictability and reliability, however, are paramount. Configuration and installation scripts need to run reliably on thousands of Macs across multiple versions of macOS (even future versions) regardless of what else is installed.
As MacAdmins, we also (should) have the tools and experience to deploy and maintain binaries in predictable locations. But then, like everyone, we only have limited time, and often need to prioritize business critical software over our own tooling. So, the pre-installed interpreter binaries have little ‘friction’ to use, even if they may have a reduced functionality when compared to the latest version available elsewhere.
This is the reason bash v3.2 is still present on macOS 12.3 and it will never be easy when Apple ultimately decides to remove it. So many tools and scripts rely on
(I don’t expect the removal to be any time soon, but there is a limit to how long Apple will or can keep this interpreter from 2007 on the system. We got the first warning when Apple switched the default interactive shell to zsh in Catalina. There will be more warnings …I hope. With the removal of the Python 2 binary we saw that Apple can move quickly when they feel the need. They did not even wait for a major macOS release.)
In this context, there is no gain in using
/usr/bin/env. The trade-offs favor the absolute shebang very strongly.
After this rant, you may think that I recommend against using
/usr/bin/env shebangs always. But there are very good use cases. Using
/usr/bin/env shebangs is the solution for workflows where cross-platform portability is required.
When your scripts need to run across multiple platforms, installing the binaries in the same location in the file system may not be possible, or require an unreasonable effort. For example, the
/usr/bin are protected by SIP and the Sealed System Volume on macOS, so you cannot add your tooling there without significantly impacting the integrity and security of the entire system.
In these cases, using a
/usr/bin/env shebang provides the required flexibility, so your scripts can work across platforms. But the portability does not come magically from just switching the shebang.
The target platforms need the binary to be installed and the versions should match. The installation location of the binary has to be present in the
PATH in the context the script runs in. To get reliable behavior, the systems you are porting between need to be well managed, with a predictable setup and configuration of the interpreter binary and environment.
When your scripts work ‘effortlessly’ across systems with the
env shebang, it is thanks to the work of the platform developers and your sysadmins/devops team for creating and maintaining this consistency. Even if you are the sole developer and devops admin, maintaining all the systems, you have to put in this work. Also the platform developers put in a lot of effort to achieve much of this consistency out of the box. As the commenter noted, some platforms don’t even agree where the
env binary should be.
You gain portability at the price of increased maintenance.
Trade-offs all the way down
Alternatively, you can keep the scripts simple – restricted to a subset of features common across platforms and versions – so that the differences have no impact. Then you trade for reliability and portability at the price of functionality.
This is often the trade-off with Python (restrict the use of python features to those common among versions) and one of the reasons Python 2 was kept around for so long on macOS.
Using POSIX sh, instead of bash or zsh, is another option to gain portability, but that has its own trade-offs. Most of these trade-offs will be in functionality, but also consider not all sh emulations are equal, and supporting multiple different emulators or the real common subset of functionality, requires extra effort.
Shebangs with absolute paths have their strengths and weaknesses, as do shebangs with
/usr/bin/env. Each has their use case and you have to understand the trade-offs when using either. Neither
env shebangs nor absolute path shebangs are generally ‘better.’ Either one may get you in trouble when used in the wrong context.
When someone says ‘you should be using,’ or ‘a better way would be,’ you always need to consider their context, use case, and which trade-offs they are accepting. You need to understand the tools to use them efficiently.