1

I'm new to linux systems and I can't really understand why wee need two operators that can redirect output: pipe as | and ouput redirection operator >? Can't we just always use the second? Most of the times I see that the pipe is used if multiple commands are chained together. If however, the output is redirected to file, as in echo 'hello' > filename, the output redirection operator is used. What am I missing here?

1
  • What its input redirection? Jun 26, 2014 at 10:34

4 Answers 4

5

I believe the < > operators are used for reading/writing files whereas the | symbol is used for piping the stdout of one command to another.

cal | less 

lets you view the output of cal in a command called less.

cal > less

puts the output of cal into a file called less.

3
  • aha, so output redirection - >- is used only to write to files? I can't chain commands using output redirection? Jun 26, 2014 at 10:40
  • @Maximus you can't do that. Jun 26, 2014 at 10:40
  • There is a tool called tee which allows you to redirect output to more than one place (stdout and a file).. or at least I think that's what it does, google it.
    – John Hunt
    Feb 20, 2023 at 20:58
4
  • | are used to send the output of one command as input to the another command which comes next after the pipe symbol.

    $ echo foo | grep -o 'f'
    f
    
  • To redirect the output of one command to a file , you may use an output redirection > operator.

    $ echo foo > file1
    

    It writes foo to file1. You don't need to manually create that file.

  • If you want to redirect the output to many files then you have to use tee command.

    echo foo | tee file1 file2
    

    It writes foo to file1 and file2. You don't need to manually create that files. Now the file1 and file2 contains only the string foo.

2
  • Extra thanks for mentioning writing to multiple files with tee Jun 26, 2014 at 10:40
  • Note that output redirection doesn't just allow you to write to files but also to things in /dev or /sys, for example, and also allows redirection from one output stream to another (eg STDERR to STDOUT, etc). Aug 20, 2018 at 1:55
3

There's a lot of talk about output redirection but I thought this question was about input. I'm going to ignore > and >> because they've nothing to do with input. Instead I'm going to focus on <, <(...) and |:

  • < expects to read from a file into STDIN while,
  • <(...) provides a file handle to the STDOUT of a command (... here)
  • | pipes STDOUT from one process into the STDIN of the next

So the < isn't directly equivalent to a pipe (it's reading from a file) and the <(...) is reading from the right place, but it's giving a file handle as an output. You need to combine them to offer an equivalent to a pipe.

a | b
< <(a) b

Just reading that, I hope that full explains why the pipe exists. It's much more readable.

5
  • Thanks a lot for your answer! Could you please explain this line < <(a) b - I didn't really understand it. Appreciate Jun 26, 2014 at 16:48
  • 1
    Sure. <(a) is taking the output of a and making it available as a file. The other < is reading that file and turning that into STDIN for b.
    – Oli
    Jun 26, 2014 at 17:04
  • Got you, thanks! Why didn't you write that way: b < <(a), or than it's going to do different thing? And what is a and b in your examples? Is a a file and b a command? Jun 27, 2014 at 5:11
  • 1
    That's exactly the same; I just left it that way around for easier comparison with the pipe. They're both commands just as they would be in a | b
    – Oli
    Jun 27, 2014 at 7:25
  • Worth noting, that underneath the hood the file offered by <(a) is actually a pipe. See Why does process substitution result in a file called /dev/fd/63 which is a pipe? Aug 20, 2018 at 1:46
2

The key point to remember is that pipes are inter-process communication device that allows two processes ( and that's what commands really are) to exchange data, while redirection operators are for manipulating where particular process writes.

In the video Unix Pipeline, the creator of awk language and one of the original people who worked on AT&T Unix Brian Kernighan explains:

First, you don't have to write one big massive program - you've got existing smaller programs that may already do parts of the job...Another is that it's possible that the amount of data you're procesing would not fit if you stored it in a file...because remember, we're back in the days when disks on these things had, if you were lucky, a Megabyte or two of data...So the pipeline never had to instantiate the whole output

As you can see, within the context which the pipelines were created, they actually were not just communication device, but also save storage space and simplify the development. Sure, we can use output/input redirection for everything (especially nowadays with storage capacity being in the range of Terabytes), however that would be inefficient from the storage point of view, and also processing speed - remember that you're directly feeding output from one command to another with |. Consider something like command1 | grep 'something'. If you write output of command1 first to a file, it will take time to write everything, then let grep go through the whole file. With pipeline and the fact that the output is buffered (meaning that left-side process pauses before right-side process is ready to read again), the output goes directly from one command to the other, saving time.

It is worth noting, that for inter-process communication, there's a use case of named pipes, to which you can use > operator to write from one command, and < to let another command read from it, and it's a use case where you do want to have particular destination on filesystem where multiple scripts/commands can write to and agree on that particular destination. But when it's unnecessary, anonymous pipe | is all you really need.

6
  • I got to use named pipes for progress bar display and yad. It was interesting the way the file is of FIFO (First In First Out) format and after you read a line it automatically disappears from the file. Aug 20, 2018 at 2:30
  • @WinEunuuchs2Unix Yep, these things have a lot of potential. And remember - they only have name on the filesystem, but otherwise serve as conduit for data and therefore won't store anything ( well, technically the kernel can store small buffered amount while the named pipe is waiting to be read, but nothing actually goes to disk - all in memory buffers). Aug 20, 2018 at 2:35
  • In most cases the progress bar is only 0 to 100 so it's relatively safe having a latency between main script continuously writing to the pipe on integer change and separate sleeping progress bar script that wakes up 4 to 10 times per second reading from the pipe. But it is good you reminded me about the cache limits in the case of a program bug and the writing script overflowing the buffer. Aug 20, 2018 at 2:42
  • Thanks a lot for you elaborate and in-depth answer, exactly what I was looking for back then. Are you on Twitter? Aug 23, 2018 at 5:40
  • @AngularInDepth I am on Twitter but don't really post anything Aug 23, 2018 at 5:45

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .