the function awk '!seen[$0]++' print the new occurrences in an input pipe.

I am however unable to add a pipe after. E.g

my_function_with_ouput | awk '!seen[$0]++' | while read j
echo $j

Doesn't produce any output.

  • I have corrected the thread – user123456 Sep 28 '16 at 11:23

awk buffers the STDOUT stream, you need to flush the STDOUT, make it flush the stream using the fflush() function of GNU awk:

my_function_with_ouput | awk '!seen[$0]++ {print; fflush()}' | while ...

If you don't have gawk, take help from stdbuf, unbuffered STDOUT:

my_function_with_ouput | stdbuf -o0 awk '!seen[$0]++' | while ...

or make STDOUT line buffered:

my_function_with_ouput | stdbuf -oL awk '!seen[$0]++' | while ...

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.