18

I have a script executing commands like:

export LD_LIBRARY_PATH=.:$LD_LIBRARY_PATH;./some_app -i $INDEX | tee $LOG
echo "Number of errors: $(grep "ERROR" $LOG | wc -l)"

The problem is probably in the pipe to tee. It does not seem to get the whole output. When the application quits last few lines of the output (usually those containing a fatal error) are missing. When I run the app without pipe to tee I get them in the output.

How can I force script to wait for tee to complete processing of all output?

Improve this question
1
  • Does it work OK if you tee it to a file, not stdout?
    – Pilot6
    Jun 23, 2015 at 10:39

2 Answers 2

Reset to default
31

The fatal error is probably coming out in STDERR (2), not STDOUT (1). You can redirect STDERR into STDOUT with 2>&1 and then the pipe should capture it too.

./some_app -i $INDEX 2>&1 | tee $LOG

If you have buffering issues on top, you could force it into an unbuffered state:

stdbuf -o0 ./some_app -i $INDEX 2>&1 | tee $LOG
Improve this answer
4
  • Good, we are getting closer. Now I see that fatal error is being printed but again it is not complete. The line with the error just ends in the middle and echo output continues. There is still some problem with flushing buffer or simply waiting on that part to complete.
    – Ladislav Mrnka
    Jun 23, 2015 at 10:33
  • 1
    Edited. Fairly rare in my experience that something so completely slips through buffers on exit but it's worth a go.
    – Oli ♦
    Jun 23, 2015 at 10:38
  • 1
    Done! Thank you. I may be asking too many question but does anyone understand why I need to turn off buffering when piping to other process?
    – Ladislav Mrnka
    Jun 23, 2015 at 10:40
  • @Oli A very good one!
    – Pilot6
    Jun 23, 2015 at 10:50
7

As the error messages are normally shown on STDERR (File descriptor 2), you need to redirect both the STDOUT and STDERR to tee:

./some_app -i "$INDEX" |& tee "$LOG"

When you do ./some_app -i $INDEX | tee $LOG you are only redirecting the STDOUT to tee.

|& will cause both the STDOUT and STDERR to be redirected.

If you cant to redirect only the STDOUT (As you were):

./some_app -i "$INDEX" | tee "$LOG"

On the other hand if you want to redirect only the STDERR:

./some_app -i "$INDEX" 2>&1 >/dev/null | tee "$LOG"
Improve this answer

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .