I've got an rdiff-backup setup that backs up a largeish sql dump (1.6GB) over a network. The server it's backing up to (Ubuntu 10.10 server) has a small local disk (4GB, 1GB free) but has a cifs mount to a large nas (plenty of free space). I've a softlink set up from a user account's home dir to the NAS
cifs mount, and the
rdiff-backup job initiated from the client is writing to that directory.
Recently, the job has been failing. The logfile says:
Sun Oct 24 21:20:14 2010 Sending back exception [Errno 28] No space left on device of type type 'exceptions.IOError'
I've run the job manually with
watch df running on the server, and I can see the 1GB of free space on the local drive diminishing until it hits zero. Then the job fails. To be clear, disk space is not an issue on the NAS.
So, for some reason, rdiff-backup is using somewhere on the local disk as a temp working directory rather than the actual directory where the backups are stored, on the NAS device, as I'd expect. However, I can't find which dir it's using so I can mount that separately with its own blob of space. Doing
du -sx on
/ while the job is running reveals the total filesize of
/ remaining constant at 3.0GB, even though
/ is full.
If I list the file handles open by the
rdiff process, I get a few sockets and some files on NAS mount. Nothing obvious on the local disk.
What's going on? Why does
/ is full, but
du -sx say there's 1GB free? Why is
rdiff-backup filling up the local disk when it should be using the
cifs mount? What's it actually writing to?