Oh, but sponge
is not the only option; you do not have to obtain moreutils
in order to get this to work properly. Any mechanism will work as long as it satisfies the following two requirements:
- It accepts the name of the output file as a parameter.
- It only creates the output file once all input has been processed.
You see, the well known problem that the OP is referring to is that the shell will create all of the files that are necessary for the pipes to work before even beginning to execute the commands in the pipeline, so it is the shell that actually truncates the output file (which unfortunately is also the input file) before any of the commands have even had a chance to start executing.
The tee
command does not work, even though it satisfies the first requirement, because it does not satisfy the second requirement: it will always create the output file immediately upon starting, so it is essentially as bad as creating a pipe straight into the output file. (It is actually worse, because its use introduces a non-deterministic random delay before the output file gets truncated, so you might think that it works, while in fact it does not.)
So, all we need in order to solve this problem is some command that will buffer all of its input before producing any output, and that is capable of accepting the output filename as a parameter, so that we do not have to pipe its output into the output file. One such command is shuf
. So, the following will accomplish the same thing that sponge
does:
shuf --output=file --random-source=/dev/zero
The --random-source=/dev/zero
part tricks shuf
into doing its thing without doing any shuffling at all, so it will buffer your input without altering it.