How to truncate a file that is being written to

Question

I have a long running process on a remote computer which I started like this:

$ nohup ./process > output &

My problem is that the output file is getting bigger and bigger fast. And of course, the only use this file has to me is monitoring the progress of the process:

$ tail -f ./output

So I would like to truncate the file from time to time to conserve the space (I'm afraid I might run out of space). So I tried this:

$ truncate -s 0 output

But it seems since the file is open and being written to, this command has no effect. As a test, once I removed the file but then there wasn't a new one created. So I had lost my progress report and I had to restart the process.

Is there any way I can truncate the file while it is being written to?

Answer 1

While a full size solution may involve logrotate, it might work with a simpler solution according to the following demo example.

I started a process, #1, that writes the current date and time to the file output.

while true;do LC_TIME=en_US date>>output;sleep 10;done &

Then I started a process, #2, that copies output, and uses tail to truncate the redirected copy back to output. This keeps writing from process #1 alive. It might work in your case too.

while true;do cat output > ttt;tail ttt > output;cat output;echo '----------------------------';sleep 60;done &

Example of output from process #2,

Sun Nov 25 14:31:40 CET 2018
----------------------------
Sun Nov 25 14:31:10 CET 2018
Sun Nov 25 14:31:20 CET 2018
Sun Nov 25 14:31:30 CET 2018
Sun Nov 25 14:31:40 CET 2018
Sun Nov 25 14:31:50 CET 2018
Sun Nov 25 14:32:00 CET 2018
Sun Nov 25 14:32:10 CET 2018
Sun Nov 25 14:32:20 CET 2018
Sun Nov 25 14:32:30 CET 2018
Sun Nov 25 14:32:40 CET 2018
----------------------------
Sun Nov 25 14:32:10 CET 2018
Sun Nov 25 14:32:20 CET 2018
Sun Nov 25 14:32:30 CET 2018
Sun Nov 25 14:32:40 CET 2018
Sun Nov 25 14:32:50 CET 2018
Sun Nov 25 14:33:00 CET 2018
Sun Nov 25 14:33:10 CET 2018
Sun Nov 25 14:33:20 CET 2018
Sun Nov 25 14:33:30 CET 2018
Sun Nov 25 14:33:40 CET 2018
----------------------------

I also tested how to make a demo example that does not hangup, if the window/connection is closed,

nohup bash -c 'while true;do LC_TIME=en_US date>>output;sleep 10;done' &

and

nohup bash -c 'while true;do cat output > ttt;tail ttt > output;sleep 60;done' &

which can be monitored with LANG=C tail -f ./output from another window/connection,

Sun Nov 25 15:01:05 CET 2018
Sun Nov 25 15:01:15 CET 2018
Sun Nov 25 15:01:25 CET 2018
Sun Nov 25 15:01:35 CET 2018
Sun Nov 25 15:01:45 CET 2018
Sun Nov 25 15:01:55 CET 2018
Sun Nov 25 15:02:05 CET 2018
tail: ./output: file truncated
Sun Nov 25 15:00:35 CET 2018
Sun Nov 25 15:00:45 CET 2018
Sun Nov 25 15:00:55 CET 2018
Sun Nov 25 15:01:05 CET 2018
Sun Nov 25 15:01:15 CET 2018
Sun Nov 25 15:01:25 CET 2018
Sun Nov 25 15:01:35 CET 2018
Sun Nov 25 15:01:45 CET 2018
Sun Nov 25 15:01:55 CET 2018
Sun Nov 25 15:02:05 CET 2018
Sun Nov 25 15:02:15 CET 2018
Sun Nov 25 15:02:25 CET 2018
Sun Nov 25 15:02:35 CET 2018
Sun Nov 25 15:02:45 CET 2018