Take the 2-minute tour ×
Ask Ubuntu is a question and answer site for Ubuntu users and developers. It's 100% free, no registration required.

I am writing data munging scripts (python and bash), to munge data and import large quantities of text files into a database.

I am currently in the test phase, so I am generating several K's of files and deleting them (the files consume about 20G of space). After a test run, I delete the files (sometimes without having imported into the database).

I notice that there is a steady decrease in the amount of free space on my disk (as reported by df -h). I don't understand this, as I use rm * (in the data directory), and in the cases where I use Nautilus, I empty the Trash bin as well.

Similarly, I notice that when I import the data into the (postgresql) database, and then delete the data from the tables using DELETE FROM tablename;, the size consumed in the postgresql data directory does not go down either.

Currently, I have lost approximately 200G from hard drive, and I need to reclaim that - but don't know what to do to reclaim it - any ideas?.

I am running Ubuntu 10.0.4 LTS + postgresql 8.4

[[Update]]

du -c / | 2>/dev/null | grep total produces this output:

4   /usr/src/linux-headers-2.6.32-38-generic/include/config/mtdram/total
4   /usr/src/linux-headers-2.6.32-41-generic/include/config/mtdram/total
4   /usr/src/linux-headers-2.6.32-36-generic/include/config/mtdram/total
4   /usr/src/linux-headers-2.6.32-30-generic/include/config/mtdram/total
4   /usr/src/linux-headers-2.6.32-35-generic/include/config/mtdram/total
4   /usr/src/linux-headers-2.6.32-31-generic/include/config/mtdram/total
4   /usr/src/linux-headers-2.6.32-27-generic/include/config/mtdram/total
4   /usr/src/linux-headers-2.6.32-28-generic/include/config/mtdram/total
4   /usr/src/linux-headers-2.6.32-40-generic/include/config/mtdram/total
4   /usr/src/linux-headers-2.6.32-34-generic/include/config/mtdram/total
4   /usr/src/linux-headers-2.6.32-37-generic/include/config/mtdram/total
4   /usr/src/linux-headers-2.6.32-33-generic/include/config/mtdram/total
4   /usr/src/linux-headers-2.6.32-39-generic/include/config/mtdram/total
4   /usr/src/linux-headers-2.6.32-29-generic/include/config/mtdram/total
4   /usr/src/linux-headers-2.6.32-32-generic/include/config/mtdram/total
187426036   total

df -hi produces this output:

Filesystem            Inodes   IUsed   IFree IUse% Mounted on
/dev/sda1                29M    1.8M     27M    7% /
none                    491K     776    490K    1% /dev
none                    492K       6    492K    1% /dev/shm
none                    492K      76    492K    1% /var/run
none                    492K       2    492K    1% /var/lock
none                    492K       1    492K    1% /lib/init/rw
/dev/sdb1                30M    3.0K     30M    1% /mydata
/dev/sdc1                  0       0       0    -  /media/Cruzer
share|improve this question

1 Answer 1

up vote 0 down vote accepted

Please show the output of du -c / | grep total and df -hi to judge where the space went. Also, refer to this post about Flushing the Postgresql cache.

EDIT: You might want to either vacuum and reindex the database, or dump and restore. The growing size on many DELETE operations is a feature, not a bug, of Postgresql. See here.

share|improve this answer
    
Please see my updated answer –  Homunculus Reticulli Jul 4 '12 at 16:08
    
Please see my updated answer to your update! –  dotancohen Jul 4 '12 at 16:31
    
Thanks - it turns out I had to vacuum the databases. –  Homunculus Reticulli Jul 4 '12 at 17:17
    
Great, I'm glad it's taken care of. Just a curiosity: why did you choose Postgresql instead of MySQL? –  dotancohen Jul 4 '12 at 18:10
    
I have always prefered PG for technical as well as business reasons. what first attracted me (about 10yrs ago) was that PG was more enterprise ready back then than MySQL. I also liked the PG License. Lastly, I have developed a lot of C extension libs that I use in my SQL queries on PG. –  Homunculus Reticulli Jul 4 '12 at 19:36

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.