You could at least start with
iotop. It won't tell you which filesystem is being written but it will give you some processes to investigate.
sudo apt-get install iotop
It shows instantaneous disk reads and writes and the name of the command reading or writing.
If you are trying to catch a process that writes infrequently, you can use the
--accumulated option or log the output to a file:
iotop --batch > iotop_log_file
Obviously the writing of the log file will show up in the results, but you should also be able to
grep for other processes writing to disk.
By this point you should be able to find some candidate suspect processes. The left column in
iotop shows the pid. Next, find out which file descriptor the process is writing to:
strace -p <pid> 2>&1 | grep write
You should see output like this when the process writes:
write(1, "\n", 1) = 1
write(4, "test\n", 5) = 5
write(1, ">>> ", 4) = 4
The first argument to write is the file descriptor. We are probably looking for values greater than 2, because 0, 1 and 2 are just stdin, stdout and stderr. File descriptor 4 looks interesting.
You can now find out what file the file descriptor points to with:
lsof -p <pid>
Which should yield output like:
python 23908 rob mem REG 8,1 26258 8392656 /usr/lib/x86_64-linux-gnu/gconv/gconv-modules.cache
python 23908 rob 0u CHR 136,5 0t0 8 /dev/pts/5
python 23908 rob 1u CHR 136,5 0t0 8 /dev/pts/5
python 23908 rob 2u CHR 136,5 0t0 8 /dev/pts/5
python 23908 rob 3w REG 0,25 909 9049082 /home/rob/testfile
python 23908 rob 4w REG 0,25 20 9049087 /home/rob/another_test_file
Look at the 4th column.
4w means that file descriptor 4 is open for writing and the file is
It is possible for a process to open, write and then close a file, in which case
lsof would not show it. You might catch this happening with:
strace -p <pid> 2>&1 | grep open