Is there any short way to save the pipe output to the same file that it's processed. For example, this is what I am actually doing

$ cat filename | sort | uniq > result
$ rm -f filename
$ mv result filename

I was wondering if there was a way to do it in just one line (not appending those commands using &&)

This is not the way, but to get an idea

$ cat filename | sort | uniq > filename

5 Answers 5


You can use sponge from moreutils package:

LC_ALL=C sort -u filename | sponge filename

You also don't need pipe to uniq, since when sort has -u option to unique lines when sorting.

Note that on GNU system with UTF-8 locales, sort -u or sort | uniq didn't give you unique lines, but the first from sequence of lines which sort the same in current locale.

$ printf '%b\n' '\U2460' '\U2461' | LC_ALL=en_US.utf8 sort | LC_ALL=en_US.utf8 uniq

gave you only . Changing locale to C force the sorting order based on the byte values:

$ export LC_ALL=C
$ printf '%b\n' '\U2460' '\U2461' | LC_ALL=C sort | LC_ALL=C uniq

You don't need any extra command like cat and uniq and also without using rm command and mv command to removing and renaming the filename. just use simple command.

sort -u filename -o filename

 -u, --unique
        with -c, check for strict ordering; without -c, output only  the
        first of an equal run

 -o, --output=FILE
        write result to FILE instead of standard output

How it work?

sort command sorts your filename and with -u option, removes duplicate lines from it. then with -o option writes the output to the same file with in place method.

  • 3
    If system crash when sort is running, you will loose your original file.
    – cuonglm
    Sep 26, 2014 at 10:20
  • 1
    thanks! in this example, with ´sort´ particularly, I should do that. However I was thinking about a general case. @Gnouc, haha there's no way to think that if it didn't happen to you, right? Sep 27, 2014 at 4:55

Your suggested example (below) doesn't work because you'd actually be reading from and writing to the same file simultaneously.

$ cat filename | sort | uniq > filename

The idea with a pipe or redirect is that the command on the left and right hand side of each pipe or redirect run simultaneously, in parallel. The command on the right processes information as it is handed over to it from the command on the left, while the command on the left is still running.

In order for your scenario to work, the command that reads from the file would need to finish before the command that writes to the file begins. In order for this to work you would need to redirect the output into a temporary location first, then once that's finished, send it from the temporary location back into the file.

A better way to do this is basically like in your former example, where you redirect to a temporary file then rename that file back to the original (except that you don't need to delete the file first, because moving deletes any existing target).

$ cat filename | sort | uniq > result
$ mv -f result filename

You could also save it into a string variable, except that only works when data is small enough to all fit in memory at once.

  • As someone put out in a suggested edit, you can change cat filename | sort to just sort filename - cat is unnecessary here. Sep 26, 2014 at 5:58
  • My example below wasn't the way to do it. Thanks for clarification. cat could be unnecessary in this case but I was focusing on the redirection part. Sep 26, 2014 at 6:04
  • 1
    I was explaining why your below example wouldn't work. I know that you knew it didn't work. Sep 26, 2014 at 6:05
  • Thanks for clarifying! In fact I didn't know what was really happening. Sep 26, 2014 at 6:08

You can use the tee command:

sort -u filename | tee filename > /dev/null

The tee command reads from standard input and writes to standard output and files.

  • 2
    This does not work for me. Oct 18, 2015 at 5:40
  • 3
    This does not work askubuntu.com/a/752451
    – Zombo
    Apr 10, 2016 at 22:52
  • This does work for me. e.g., to move a line to the bottom of a file: (cat ~/file | grep -v 3662 ; printentry 3662) | tee ~/file > /dev/null works. Like the original post, this doesn't work if you just > ~/file without the tee. Tee seems similar here to sort -o file, which writes to the named file without continuing the same pipe. Apr 3, 2017 at 4:56
  • Wait, sorry! I've seen empirically this will unpredictably lose data as explained in the link from @Steven. Make a file with the numbers 1..9 on 9 lines. The following will work several times, and then occasionally remove all of the data from the file: (cat x | grep -v 7 ; echo 7) | tee x > /dev/null; cat x I recommend a temp file and mv or perhaps the solution from @Steven's link. Apr 20, 2017 at 16:13
  • @JoshuaGoldberg have you seen my answer on this page?
    – Zombo
    Apr 20, 2017 at 16:20

You can use Vim in Ex mode:

ex -sc 'sort u|x' filename
  1. sort u sort unique

  2. x write if changes have been made (they have) and quit

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .