3

I would like to do something equivalent to this:

some-expensive-command > /tmp/mytempfile
grep -v "pattern" /tmp/mytempfile >> output.txt
grep "pattern" /tmp/mytempfile | yet-another-command

Preferably elegant and without the need for the tempfile. I was thinking about piping through tee, but the best I can think of might is to combine two of the three lines and still require the intermediate storage:

some-expensive-command | tee /tmp/mytempfile | grep -v "pattern" >> output.txt
grep "pattern" /tmp/mytempfile | yet-another-command
  • So you want one command's output saved to file output.txt and that same output redirected to another command for further processing ? Is that what you're trying to do ? – Sergiy Kolodyazhnyy Nov 17 '18 at 22:29
3

The way question reads it sounds like you want one stdin redirected to two different commands. If that's the case, take advantage of tee plus process substitution:

some-expensive-command | tee >(grep 'pattern' > output.txt) >(grep -v 'pattern' | another-command)

Process substitutions are in fact anonymous pipelines implemented within bash itself ( on subprocess level ). We can also make the use of a named pipeline + tee. For instance, in terminal A do

$ mkfifo named.fifo
$ cat /etc/passwd | tee named.fifo | grep 'root'

And in another terminal B do

$ grep -v 'root' named.fifo

Another way to look at this is by recognizing that grep is line pattern matching tool, so by reading line at a time and using that same line in multiple commands we can achieve exactly the same effect:

rm output.txt # get rid of file so that we don't add old and new output
some-expensive-command | while IFS= read -r line || [ -n "$line" ]; do
    printf "%s\n" "$line" | grep 'pattern' >> output.txt
    printf "%s\n" "$line" | grep -v 'pattern' | another-command
done
# or if another-command needs all of the output, 
# place `| another-comand` after `done` clause

Yet another way is to abandon grep and use something more powerful, like awk:

some-expensive-command | awk '/pattern/{print >> "output.txt"}; !/pattern/{print}' | another-command.

Practically speaking, don't worry about using temporary files, so long as you clean them up after using. If it works, it works.

| improve this answer | |
1

Use bash Process Substitution:

some-command | tee >(grep "pat" | another-command >>out1) | grep -v "pat" >>out2

The process substitution assigns some-command’s output to grep "pat"’s input, thus saving you the tempfile. Of course the data is still saved in a file (it’s always), just that you don’t have to take care of that. If you don’t want to save another-command’s output in a file but rather print it I recommend to simply switch the two command lists.

Another nice source of information: man bash/EXPANSION

| improve this answer | |

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.