73

In bash or sh, I guess anything which starts with # is a comment.

But in bash scripts we write:

#!/bin/bash

And in Python scripts, there is:

#!/bin/python

Does this mean that # by itself is a comment whereas #! is not?

2
  • 1
    And once you start looking at Apparmor profiles, you'll see #include. There too, the # isn't meant as a comment.
    – user25656
    Jan 9, 2013 at 6:07
  • 4
    @vasa1 But the key point that's often not appreciated about hashbang lines at the beginning of shell scripts is that they are comments. Jan 9, 2013 at 7:33

4 Answers 4

110

The #! line is used before the script is run, then ignored when the script runs.

You're asking what the difference is between a shebang line and an ordinary comment.

A line starting with #! is just as much a comment as any other line that starts with #. This is true if the #! is the first line of the file, or anywhere else. #!/bin/sh has an effect, but it is not read by the interpreter itself.

# is not a comment in all programming languages but, as you know, it's a comment in Bourne-style shells including sh and bash (as well as most non-Bourne-style shells, like csh). It's also a comment in Python. And it is a comment in a variety of configuration files that are not really scripts at all (like /etc/fstab).

Suppose a shell script begins with #!/bin/sh. That is a comment, and the interpreter (the shell) ignores everything on the line after the # character.

The purpose of a #! line is not to give information to the interpreter. The purpose of the #! line is to tell the operating system (or whatever process launches the interpreter) what to use as the interpreter.

  • If you invoke the script as an executable file, for example, by running ./script.sh, the system consults the first line to see if begins with #!, followed by zero or more spaces, followed by a command. If it does, it runs that command with the name of the script as its argument. In this example, it runs /bin/sh script.sh (or, technically, /bin/sh ./script.sh).

  • If you invoke the script by explicitly calling the interpreter, the #! line is never consulted. So, if you run sh script.sh, the first line has no effect. If script2.sh's first line is #!/usr/games/nibbles, running sh script2.sh will not try to open the script in nibbles (but ./script2.sh will).

You'll notice that in neither case does the script's extension (.sh), if it has one, impact how it is run. In a Unix-like system, this doesn't ordinarily affect how the script is run. On some other systems, like Windows, the #! shebang line might be ignored entirely by the system, and the extension might determine what runs the scripts. (This doesn't mean you need to give your scripts extensions, but it is one of the reasons why if you do, they should be correct.)

#! was chosen to serve this purpose precisely because # begins a comment. The #! line is for the system, not the interpreter, and it should be ignored by the interpreter.

Shebang Line for Bash Scripts

You (originally) said you use #!/bin/sh for bash scripts. You should only do that if the script doesn't require any of bash's extensions--sh needs to be able to run the script. sh is not always a symlink to bash. Often, including on all remotely recent Debian and Ubuntu systems, sh is a symlink to dash.

Shebang Line for Python Scripts

You also said (in the first version of your question, before editing) that you start your Python scripts with #!/bin/sh read by the interpretor. If you mean that literally, then you should definitely stop doing that. If hello.py begins with that line, running ./hello.py executes:

/bin/sh read by the interpretor hello.py

/bin/sh will try to execute a script called read (with by the interpretor hello.py as its arguments), read will (hopefully) not be found, and your Python script will never be seen by a Python interpreter.

If you're making this mistake but not having the problem I'm describing, you're probably invoking your Python scrips by explicitly specifying the interpreter (e.g., python hello.py), causing the first line to be ignored. When you distribute your scripts to others, or use them a long while later, it may not be clear that this is necessary for them to work. It's best to fix them now. Or at least remove the first line entirely, so that when they fail to run with ./ the error message will make sense.

For Python scripts, if you know where the Python interpreter is (or is going to be), you can write the #! line the same way:

#!/usr/bin/python

Or, if it's a Python 3 script, you should specify python3, since python is almost always Python 2:

#!/usr/bin/python3

However, the problem is that while /bin/sh is supposed to always exist, and /bin/bash almost always exists on systems where bash comes with the OS, Python may exist in a variety of places.

Therefore, many Python programmers use this instead:

#!/usr/bin/env python

(Or #!/usr/bin/env python3 for Python 3.)

This makes the script rely on env being in the "right place" instead of relying on python being in the right place. That's a good thing, because:

  • env is almost always located in /usr/bin.
  • On most systems, whichever python should run your script is the one that appears first in the PATH. Starting hello.py with #!/usr/bin/env python make ./hello.py run /usr/bin/env python hello.py, which is (virtually) equivalent to running python hello.py.

The reason you cannot use #!python is that:

  • You want the interpreter specified to be given by an absolute path (i.e., starting with /).
  • The calling process would execute python in the current directory. Searching the path when the command doesn't contain a slash is specific shell behavior.

Occasionally a Python or other script that isn't a shell script will have a shebang line starting with #!/bin/sh ... where ... is some other code. This is sometimes correct, because there are some ways to invoke the Bourne-compatible shell (sh) with arguments to make it invoke a Python interpreter. (One of the arguments will probably contain python.) However, for most purposes, #!/usr/bin/env python is simpler, more elegant, and more likely to work the way you want.

Shebang Lines in Other Languages

Many programming and scripting languages, and some other file formats, use # as a comment. For any of them, a file in the language can be run by a program that takes it as an argument by specifying the program on the first line after #!.

In some programming languages, # is not normally a comment, but as a special case the first line is ignored if it starts with #!. This facilitates the use of #! syntax even though # doesn't otherwise make a line a comment.

Shebang Lines for Files That Don't Run As Scripts

While it's less intuitive, any file whose file format can accommodate a first line starting with #! followed by the full path of an executable can have a shebang line. If you do this, and the file is marked executable, then you can run it like a program...causing it to be opened like a document.

Some applications use this behavior intentionally. For example, in VMware, .vmx files define virtual machines. You can "run" a virtual machine as though it were a script because these files are marked executable and have a shebang line causing them to be opened in a VMware utility.

Shebang Lines for Files That Don't Run as Scrips but Act Like Scripts Anyway

rm removes files. It is not a scripting language. However, a file that starts #!/bin/rm and is marked executable can be run, and when you run it, rm is invoked on it, deleting it.

This is often conceptualized as "the file deletes itself." But the file is not really running at all. This is more like the situation described above for .vmx files.

Still, because the #! line facilitates the running of a simple command (including command-line arguments), you can perform some scripting this way. As a simple example of a "script" more sophisticated than #!/bin/rm, consider:

#!/usr/bin/env tee -a

This takes user input interactively, echoes it back to the user line-by-line, and appends it to the end of the "script" file.

Useful? Not very. Conceptually interesting? Totally! Yes. (Somewhat.)

Conceptually Similar Programming/Scripting Concepts (just for fun)

7
  • @Rinzwind Thx! (Btw this answer doesn't originate elsewhere, if that's what you're wondering.) Jan 9, 2013 at 8:28
  • @Rinzwind Don't worry, with 8 upvotes after 1 hour it is likely to raise much further :-)
    – guntbert
    Jan 9, 2013 at 8:53
  • 1
    If it's always ignored, then what does Pythons -x flag do?
    – gerrit
    Jan 9, 2013 at 9:08
  • 4
    @gerrit Good question. In any language where the intepreter/compiler reports messages with line numbers, comments' contents are ignored, but comment-lines are still counted. Adding a comment or blank line before a line of code still results in that line of code having its line number incremented. -x "skip[s] the first line..." the 2nd line gets numbered 1 instead of 2, the 3rd line 2 instead of 3, etc. This is why you shouldn't use that flag. ;) -x is for scripting on non-Unix-like OSes that have shebang-like syntax not starting with # (thus not a Python comment). Jan 9, 2013 at 9:22
  • 4
    In Perl, if the interpreter is started directly (perl script.pl vs. ./script.pl) then the interpreter will read the shebang line to parse flags such as -w. It's not recommended to rely on this feature though.
    – OrangeDog
    Jan 9, 2013 at 9:36
7

A shebang is the character sequence consisting of the characters number sign and exclamation mark (e.g. "#!") when it occurs as the initial two characters on the initial line of a script.

Under *nix operating systems, when a script starting with a shebang is run, the program loader parses the rest of the script's initial line as an interpreter directive; the specified interpreter program is run instead, passing to it as an argument the path that was initially used when attempting to run the script. For example, if a script is named with the path "path/to/your-script", and it starts with the following line:

#!/bin/sh

then the program loader is instructed to run the program "/bin/sh" instead e.g. the Bourne shell or a compatible shell, passing "path/to/your-script" as the first argument.

Accordingly, it a script is named with the path "path/to/python-script" and it starts with the following line:

#!/bin/python

then the program loaded is instructed to run the program "/bin/python" instead e.g. Python interpreter, passing "path/to/python-script" as the first argument.

In short "#" will comment out a line while the character sequence "#!" occurring as the first two characters on the initial line of a script has meaning outlined as above.

For details see Why do some scripts start with #! ... ?

Source: Some sections of this answer are derived (with slight modification) from Shebang (Unix) on English Wikipedia (by Wikipedia contributors). This article is licensed under CC-BY-SA 3.0, same as user content here on AU, so this derivation is permitted with attribution.

5

#! is called the shebang when it occurs as the initial two characters on the initial line of a script. It is used in scripts to indicate an interpreter for execution. The shebang is for the operating system(kernel), not for the shell; so it won't be interpreted as a comment.

Courtesy: http://en.wikipedia.org/wiki/Shebang_%28Unix%29

In general, if a file is executable, but actually not an executable (binary) program, and such a line is present, the program specified after #! is started with the scriptname and all its arguments. These two characters # and ! have to be the first two bytes in the file!

Detailed Info: http://wiki.bash-hackers.org/scripting/basics#the_shebang

0

No, it is only used by the exec system call of the Linux kernel, and treated as a comment by the interpreter

When you do on bash:

./something

on Linux, this calls the exec system call with the path ./something.

This line of the kernel gets called on the file passed to exec: https://github.com/torvalds/linux/blob/v4.8/fs/binfmt_script.c#L25

if ((bprm->buf[0] != '#') || (bprm->buf[1] != '!'))

It reads the very first bytes of the file, and compares them to #!.

If the comparison is true, then the rest of the line is parsed by the Linux kernel, which makes another exec call with path /usr/bin/env python and current file as the first argument:

/usr/bin/env python /path/to/script.py

and this works for any scripting language that uses # as a comment character.

And yes, you can make an infinite loop with:

printf '#!/a\n' | sudo tee /a
sudo chmod +x /a
/a

Bash recognizes the error:

-bash: /a: /a: bad interpreter: Too many levels of symbolic links

#! just happens to be human readable, but that is not required.

If the file started with different bytes, then the exec system call would use a different handler. The other most important built-in handler is for ELF executable files: https://github.com/torvalds/linux/blob/v4.8/fs/binfmt_elf.c#L1305 which checks for bytes 7f 45 4c 46 (which also happens to be human readable for .ELF). Let's confirm that by reading the 4 first bytes of /bin/ls, which is an ELF executable:

head -c 4 "$(which ls)" | hd 

output:

00000000  7f 45 4c 46                                       |.ELF|
00000004                                                                 

So when the kernel sees those bytes, it takes the ELF file, puts it into memory correctly, and starts a new process with it. See also: https://stackoverflow.com/questions/8352535/how-does-kernel-get-an-executable-binary-file-running-under-linux/31394861#31394861

Finally, you can add your own shebang handlers with the binfmt_misc mechanism. For example, you can add a custom handler for .jar files. This mechanism even supports handlers by file extension. Another application is to transparently run executables of a different architecture with QEMU.

I don't think POSIX specifies shebangs however: https://unix.stackexchange.com/a/346214/32558 , although it does mention in on rationale sections, and in the form "if executable scripts are supported by the system something may happen".

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .