17

Using process substitution or by grouping & piping , one can pass the output (stdout) a set of commands as the input for another command.

Example :

wc < <(echo hi;echo bye)
(echo hi; bye) | wc

But is there the opposite ?

That is, Is there a way to send the output of one command as the input for many other commands in a single line ?

Something like :

cat testfile | wc | tail -n 5

(Don't take pipelines literally) . Here I am trying to pass the output of cat testfile to both wc and tail -n 5

Another example would be :

(to get the last word of a sentence)

echo "some random words"| wc -c | cut -f(out put of wc -c) (output of first echo)
19

Quick answer. You can use tee >(what_to_do) >(another_thing_to_do) to keep going with your command for as many different things you want to do.

Example:

Original test file output:

:~$ cat testfile.txt 
Device Model:     LITEONIT LCS-256M6S 2.5 7mm 256GB
Serial Number:    TW0XFJWX550854187616

Output with tee commands added:

:~$ cat testfile.txt | tee >(tail -1) >(wc) >(awk '{print $3,$1,$2}')
Device Model:     LITEONIT LCS-256M6S 2.5 7mm 256GB
Serial Number:    TW0XFJWX550854187616
LITEONIT Device Model:
TW0XFJWX550854187616 Serial Number:
      2      10      91
Serial Number:    TW0XFJWX550854187616

Each command in tee is just normal commands you would use on the command line, like add in >(head -1 | wc) works as well.

:~$ cat testfile.txt | tee >(tail -1) >(head -1 | wc)  >(awk '{print $3,$1,$2}')
Device Model:     LITEONIT LCS-256M6S 2.5 7mm 256GB
Serial Number:    TW0XFJWX550854187616
      1       7      52
LITEONIT Device Model:
TW0XFJWX550854187616 Serial Number:
Serial Number:    TW0XFJWX550854187616

Or you can also grab the last word of say the last line by using awk with $NF with a wc as well like this:

:~$ cat testfile.txt | tail -1 | tee >(wc) >(awk '{print $NF}')
Serial Number:    TW0XFJWX550854187616
TW0XFJWX550854187616
      1       3      39

NOTE: Adding a | pipe command to the end can override using the multiple commands from the tee command. I have some examples here that I have been testing:

Example 1 (Pipe command pulling all last words):

:~$ echo "This is just five words" | tee >(wc -l) >(wc -w) >(wc -c) | awk '{print $NF}'
words
24
5
1

Example 2 (Does not show the output of the wc commands. Pipe command grabbing 3rd word.):

:~$ echo "This is just five words" | tee >(wc -l) >(wc -w) >(wc -c) | awk '{print $3}'
just

Example 3 (Grabbing the 3rd word of echo line. Tee command.):

:~$ echo "This is just five words" | tee >(wc -l) >(wc -w) >(wc -c) >(awk '{print $3}')
This is just five words
just
24
5
1

Example 4 (Grabbing the last word of the echo line. Tee command.):

:~$ echo "This is just five words" | tee >(wc -l) >(wc -w) >(wc -c) >(awk '{print $NF}')
This is just five words
words
24
5
1

Hope this helps!

  • 3
    I love tee and coffee! '-) – Elder Geek Oct 4 '16 at 18:55
  • 1
    The statement about the ordering of the output between the various consumers of tee is false. The reason you usually get the correct order is that on a low load machine, the first consumer process gets to run first. On a busier machine, the first consumer might be put to sleep before it gets to print anything. – Matei David Oct 4 '16 at 18:58
  • @MateiDavid I can't prove it otherwise, so I just removed the statement. No biggie. – Terrance Oct 4 '16 at 19:05
  • 2
    See my answer. But more seriously, tee has no idea about the consumers- they are created by bash who processes the line. All that bash does is to copy the open file descriptors. Beyond that, you have several consumers writing to file descriptors opened on the same file. The kernel takes care of that, but certainly the order they appeared on the bash command line won't be a factor. – Matei David Oct 4 '16 at 19:05
17

You need tee to split the stream into parts. Try:

cat testfile | tee >(wc -l) >(wc -w) >(wc -c) | tail -n 5

Notes:

  • If multiple processes (wc, tail) are all writing to stdout:

    • You might get garbled output.

    • There is no guarantee about the order in which their output will appear. To see this, try sleep 1; wc -w as the second consumer.

  • tee will block if any of its destinations does not consume the stream fast enough. Meaning, the destinations will be fed input at roughly similar speed (modulo the fixed-size buffering). There is no easy fix for this, the only alternative is to save the stream to a file, and feed it to the consumers separately. For wc and tail this is not an issue.

For the last word, it's simpler to do:

echo "some random words" | awk '{print $NF}'
4

The moreutils package provides the command pee (merge of pipe and tee, what did you think?) which does exactly that.

For your first example, you'd use:

cat testfile | pee wc "tail -n 5"

The second example is more complicated because you want to pass two inputs to the last command. I would probably use awk as the other answers.

Link: https://joeyh.name/code/moreutils/

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.