9

I tried to rev a file and then pipe it to cat > same_file but it was turning into a blank file.

While I tried rev file.txt | cat > file2.txt && mv file.txt file2.txt; it worked.

Even rev file.txt | cat >> file.txt; worked.

But when I tried rev file.txt | cat > file.txt it failed.

  • You can also leave cat out like this: rev file.txt > file2.txt && mv file2.txt file.txt. This is a superfluous use of cat. By leaving it out you spare spawning an extra process. – matega Jan 11 '15 at 20:41
18

The basic thing you need to grasp in this case between both ways of redirecting (> and >>) is:

>

Redirects and overwrites the information where it was pointed to. This happens while receiving any information through pipe "|"

>>

Redirects and concatenates to the information it was pointed to. This happens while receiving any information through pipe "|"

In both cases if the file does not exist, it will be created instead. Only on ">>" will the information be concatenated if you run it again on the same file. With ">" you would simply overwrite everything you did on the first run.

But here is the deal when using the same input file as the output file. In that particular case, if you use ">" you would be removing the information the "input" part needs to parse, since the output file would be "overwriting it". So in:

rev file.txt | cat > file.txt

What is actually happening in "slow motion explanation" is:

  1. rev prepares to reverse the content of file.txt and send it to pipe
  2. While rev is sending the information to pipe, pipe streams it directly to cat.
  3. While cat is receiving the information it will automatically apply it to the file.txt it was set with.
  4. The keyword here is "while", because everything is happening at the same time. Please see the excellent comments below by Emil to have a deeper understanding on this part.
  5. cat will not wait for rev to pipe the whole file. It will simply start the minute the first part of the information gets to it, which means, depending on what symbol you used, it will open a connection to file.txt.
  6. In this case since you used the > instead of >>, the shell will truncate the output file which means it will open and clear the information of file.txt while waiting for the new information to get to it. With >> it would open a connection with file.txt and wait for new information on the last line detected.
  7. Since the information was already cleared in file.txt with >, rev would try to do it's work and get nothing because cat deleted everything in preparation for the new information.

So why do the others work after reading the above. Because of this:

rev file.txt | cat > file2.txt && mv file.txt file2.txt

Here you are piping to cat which is sending the information to another file. In this case, the processed input file file.txt is not the same as the output file file2.txt. After that you are literally overwriting the whole file2.txt with file.txt, so all the process made by cat was deleted. Basically the whole line could be simplified like cp file.txt file2.txt because it is doing the same thing since file2.txt at the end looses the rev and is overwritten with the mv command.

rev file.txt | cat >> file.txt

In this case you are concatenating the information to the same file. So it is only opening a connection to that file but not erasing the information as seen with a single >. The end result should be, the original information plus the reversed information.

  • 5
    The file is not truncated by cat. It is truncated by the shell, before any of the commands in the pipeline are even started. – Emil Jeřábek supports Monica Jan 11 '15 at 15:34
  • Correct, was looking for easier words to explain. Since it is a bit hard to explain if the OP does not know what a shell it and all that. Trying to make it as "friendly" as possible. – Luis Alvarado Jan 11 '15 at 15:36
  • 1
    Well, it doesn’t really matter that it is done by the shell, but that the timing you present is wrong. The truncation will not happen in step 6, but in step 0. rev file.txt | cat --bogus-option > file.txt will also truncate the file, even though cat will not attempt to open it. – Emil Jeřábek supports Monica Jan 11 '15 at 16:10
  • @EmilJeřábek you are correct. Still users that do not know how it behaves will have it easier if we go step by step with only the commands. Also your example is still sending the information to the same file, so bash will read that whole line, see the output and still open AND truncate it. The error is simply the stderr outputting to stdout. – Luis Alvarado Jan 11 '15 at 18:51
  • 4
    See also moreutils, a fantastic collection of tools (available from the package repository as moreutils) which includes sponge, a tool specifically designed for the usecase of overwriting an input file. For example, the rev file.txt >file2.txt && mv file2.txt file.txt workaround would become rev file.txt | sponge file.txt, which will work correctly even when there is already something named file2.txt. – Daniel Wagner Jan 11 '15 at 20:10
9

When the shell sees redirection, it opens the relevant files first, before executing any of the commands involved. Thus, when you do:

foo file.txt | bar > file.txt

The redirection to file.txt causes it to be truncated before foo is run and can read file.txt. On a side note, this is why you cannot do:

sed 'blah' file.txt > file.txt

And why sed has an in-place editing option.

Lastly, doing:

.. | cat > file.txt

is a useless use of cat, especially so if you are attempting to read from file.txt earlier on.

If you want to reverse a file in-place, there are no shortcuts. You might be able to use the sed or awk tricks with in-place editing.

3

> is a redirector (operator) sending the output to something else
(input of next command, printer..)

In your case the output goes to a file file.txt, if this file already exist it is overwritten, if not it is created.

>> is an append operator, if file.txt already exist the output is appended to the end of the file. if the file doesn't exist it is created, and the output written to the new file, same as > (redirector).

  • The OP seems to have understood this. The confusion seems to stem only from the same file being on both sides of the > and >> operators. – bzlm Jan 12 '15 at 12:29
0

You can use Vim in Ex mode:

ex -sc '%!rev' -cx file.txt
  1. % select all lines

  2. ! run command

  3. x save and close

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.