10

This question already has an answer here:

So I am trying to learn how to pass standard out and standard err to various areas.

Let's say I have a folder with here.txt alone.

So if I do

ls here.txt not-here.txt  1>out  2>&1

Since here.txt is present I'll have some output to direct to the file out but since not-here.txt is not present, an error will get sent through standard error, which I am redirecting to standard out with 2>&1.

However, why doesn't this work:

ls here.txt not-here.txt 2>&1 1>out

It seems to only work if I do the redirection after the standard out instruction? Why?

marked as duplicate by Zanna command-line Sep 30 '17 at 13:33

This question has been asked before and already has an answer. If those answers do not fully address your question, please ask a new question.

5

The order of redirections is significant. For example, the command

ls > dirlist 2>&1

directs both standard output and standard error to the file dirlist, while the command

ls 2>&1 > dirlist

directs only the standard output to file dirlist, because the standard error was duplicated from the standard output (typically still pointing to a terminal window) before the standard output was redirected to dirlist.

At first this may feel counter-intuitive, but after thinking about it, we can understand it.


You find this explanation in man bash, in the chapter about redirection,

REDIRECTION

Before a command is executed, its input and output may be redirected using a special notation interpreted by the shell. Redirection allows commands file handles to be duplicated, opened, closed, made to refer to different files, and can change the files the command reads from and writes to. Redirection may also be used to modify file handles in the current shell execution environment. The following redirection operators may precede or appear anywhere within a simple command or may follow a command. Redirections are processed in the order they appear, from left to right.

Each redirection that may be preceded by a file descriptor number may instead be preceded by a word of the form {varname}. In this case, for each redirection operator except >&- and <&-, the shell will allocate a file descriptor greater than or equal to 10 and assign it to varname. If >&- or <&- is preceded by {varname}, the value of varname defines the file descriptor to close.

In the following descriptions, if the file descriptor number is omitted, and the first character of the redirection operator is <, the redirection refers to the standard input (file descriptor 0). If the first character of the redirection operator is >, the redirection refers to the standard output (file descriptor 1).

The word following the redirection operator in the following descriptions, unless otherwise noted, is subjected to brace expansion, tilde expansion, parameter and variable expansion, command substitution, arithmetic expansion, quote removal, pathname expansion, and word splitting. If it expands to more than one word, bash reports an error.

Note that the order of redirections is significant. For example, the command

ls > dirlist 2>&1

directs both standard output and standard error to the file dirlist, while the command

ls 2>&1 > dirlist

directs only the standard output to file dirlist, because the standard error was duplicated from the standard output before the standard output was redirected to dirlist.

Edit: The following command lines might explain what happens

Prepare

sudodus@xenial32:~$ touch qwerty;rm asdf
rm: cannot remove 'asdf': No such file or directory

Run the list command for one existing file and one non-existing file

sudodus@xenial32:~$ ls qwerty asdf
ls: cannot access 'asdf': No such file or directory
qwerty

Redirect the error output before redirecting the standard output. Only the standard output is redirected to the output file.

sudodus@xenial32:~$ ls qwerty asdf 2>&1 > output-file ;echo '---';cat output-file 
ls: cannot access 'asdf': No such file or directory
---
qwerty

Redirect the error output after redirecting the standard output. Both the error output and the standard output are redirected to the output file.

sudodus@xenial32:~$ ls qwerty asdf > output-file 2>&1 ;echo '---';cat output-file
---
ls: cannot access 'asdf': No such file or directory
qwerty

The token &> can be used to redirect both the standard error and standard output. It can be used in bash, but may not be available in other shells.

sudodus@xenial32:~$ ls qwerty asdf &> output-file ;echo '---';cat output-file
---
ls: cannot access 'asdf': No such file or directory
qwerty
sudodus@xenial32:~$ 
  • Please only use code formatting for text that is code (i. e. not natural language) or that really requires white-space preservation. If you want to quote text passages please format those as block quotes. Otherwise +1. – David Foerster Sep 24 '17 at 19:58
  • Like if I redirect 2 to 1 it's really more like redirecting to whatever 1 is pointed to at that time and does not "follow" 1 if I redirect 1 somewhere else later? How do I redirect back to terminal and a file for instance? – AJJ Sep 24 '17 at 21:47
  • Your last comment is confusing however, I'll base my comment on what I think you mean. – jpezz Sep 25 '17 at 0:35
  • 1
    The redirection is applicable only for the one command so the next command would not be redirected. Group commands and surround them with parentheses BEFORE the redirection to make the output of all the commands be redirected. Default output if not redirected is /dev/tty [always your terminal] to your shell. To make the ouput go to more than one place, pipe the command to tee which sends it to the file specified as well as the screen. "ls | tee outfile" sends the output to the pipe & tee sends it to 2 places – jpezz Sep 25 '17 at 0:42
  • Yes, @ArukaJ, if I redirect 2 to 1 it's really redirecting to whatever 1 is pointed to 'at that time'. – sudodus Sep 25 '17 at 1:23
3
  • 2>x means that filename x will receive the data written to descriptor 2 (also known as stderr, standard error)
  • ... but when x is specified as &1 it doesn't mean "always follow 1"; it means "copy the current properties of 1 (and then leave it alone)".
  • Redirections are applied in the same order as entered on the command line, but before the actual execution happens.

This is why 2>&1 1>whatever outputs stderr to terminal.

This is why find / -name mylostfile.txt 3>&1 1>&2 2>&3 | grep -v 'Permission denied' exchanges stderr and stderr, so that you can filter some common stderr lines out, but still see all the stdout. (Descriptor 3 here is unused by programs).

  • Iow, it has value assignment semantics, not reference semantics. – JDługosz Sep 25 '17 at 5:12
2

The shell encounters and sets in the order it sees thing. In the first case:

ls here.txt not-here.txt  1>out  2>&1

The output is redirected, then standard error is sent to the same place.

In the second case,

ls here.txt not-here.txt 2>&1 1>out

Standardout out is still set to the terminal, so standard error is sent to the terminal THEN standard out is changed. The shell has already set standard error.

Not the answer you're looking for? Browse other questions tagged or ask your own question.