I have a script running which pipes its output through another script; that wrapper script adds a timestamp to each line of the log. However the data is being written to the log file in chunks of about 8000 characters (roughly 180 lines). This is causing all of those lines to be timestamped exactly the same. If the underlying script that is outputting info is run directly in the shell then the outputs appear line by line in real time.
For reference the script looks like this:
#!/bin/bash python foo.py | ~/timestamp.sh >> ~/logs/foo.log exit
The timestamp script just adds a, you guessed it, timestamp to every line of output. That script contains the following:
#!/bin/bash while read x; do echo -n `date +%d/%m/%Y\ %H:%M:%S`; echo -n " "; echo $x; done
If I remove the timestamp.sh from the equation the behaviour is exactly the same. I checked this by running the script and monitoring the log file using
tail -f script.log
Is there a setting that buffers file output before writing it to log files? Are there other methods I can try to timestamp each line of a log file? I have had this type of thing work before on previous systems but no matter what I try nothing seems to be sorting this.
This is all on Ubuntu 14.04.4 x64