Take the 2-minute tour ×
Ask Ubuntu is a question and answer site for Ubuntu users and developers. It's 100% free, no registration required.

So a client of mine got an email from Linode today saying their server was causing Linode's backup service to blow up. Why? Too many files. I laughed and then ran:

# df -ih
Filesystem     Inodes IUsed IFree IUse% Mounted on
/dev/xvda        2.5M  2.4M   91K   97% /

Crap. 2.4million inodes in use. What the hell has been going on?!

I've looked for the obvious suspects (/var/{log,cache} and the directory where all the sites are hosted from) but I'm not finding anything really suspicious. Somewhere on this beast I'm certain there's a directory that contains a couple of million files.

For context one my my busy servers uses 200k inodes and my desktop (an old install with over 4TB of used storage) is only just over a million. There is a problem.

So my question is, how do I find where the problem is? Is there a du for inodes?

share|improve this question
1  
See Where are all my inodes being used? –  gertvdijk Jul 3 '13 at 19:35
1  
run vmstat -1 100 and show us some of that. beware of large number in CS (Context switching). Sometimes a failing filesystem can loose alot of inodes to errors. Or perhaps legitimately, there are many files. This link should inform you about files and inodes. stackoverflow.com/questions/653096/howto-free-inode-usage you may need to see what is running / open with lsof command. –  j0h Jul 3 '13 at 19:51
add comment

4 Answers

up vote 7 down vote accepted

Check /lost+found in case there was a disk problem and a lot of junk ended up being detected as separate files, possibly wrongly.

Check iostat to see if some application is still producing files like crazy.

find / -xdev -type d -size +100k will tell you if there's a directory that uses more than 100kB of disk space. That would be a directory that contains a lot of files, or contained a lot of files in the past. You may want to adjust the size figure.

I don't think there's a combination of options to GNU du to make it count 1 per directory entry. You can do this by producing the list of files with find and doing a little bit of counting in awk. Here is a du for inodes. Minimally tested, doesn't try to cope with file names containing newlines.

#!/bin/sh
find "$@" -xdev -depth | awk '{
    depth = $0; gsub(/[^\/]/, "", depth); depth = length(depth);
    if (depth < previous_depth) {
       # A non-empty directory: its predecessor was one of its files
       total[depth] += total[previous_depth];
       print total[previous_depth] + 1, $0;
       total[previous_depth] = 0;
    }
    ++total[depth];
    previous_depth = depth;
}
END { print total[0], "total"; }'

Usage: du-inodes /. Prints a list of non-empty directories with the total count of entries in them and their subdirectories recursively. Redirect the output to a file and review it at your leisure. sort -k1nr <root.du-inodes | head will tell you the biggest offenders.

share|improve this answer
    
The script give errors: awk: line 2: find: regular expression compile failed (bad class -- [], [^] or [) [^ awk: line 2: syntax error at or near ] `/tmp/tmpw99dhs': Permission denied –  Radu Rădeanu Jul 4 '13 at 7:52
    
@RaduRădeanu Ah, I see, I used a gawk peculiarity that doesn't work in other versions. I've added a backslash which I think is necessary as per POSIX. –  Gilles Jul 4 '13 at 8:08
add comment

You can check with this script:

#!/bin/bash

if [ $# -ne 1 ];then
  echo "Usage: `basename $0` DIRECTORY"
  exit 1
fi

echo "Wait a moment if you want a good top of the bushy folders..."

find "$@" -type d -print0 2>/dev/null | while IFS= read -r -d '' file; do 
    echo -e `ls -A "$file" 2>/dev/null | wc -l` "files in:\t $file"
done | sort -nr | head | awk '{print NR".", "\t", $0}'

exit 0

This prints the top 10 subdirectories by file count. If you want a top x, change head with head -n x, where x is a natural number bigger than 0.

For 100% sure results, run this script with root privileges:

top-bushy-folders

share|improve this answer
add comment

Another suggest:

http://www.iasptk.com/20314-ubuntu-find-large-files-fast-from-command-line

Use these searches to find the largest files on your server.

Find files over 1GB

sudo find / -type f -size +1000000k -exec ls -lh {} \;

Find files over 100MB

sudo find / -type f -size +100000k -exec ls -lh {} \;

Find files over 10MB

sudo find / -type f -size +10000k -exec ls -lh {} \;

The first part is the find command using the "-size" flag to find files over different sizes measured in kilobytes.

The last bit on the end starting with "-exec" allows to specify a command we want to execute on each file we find. Here the "ls -lh" command to include all the information seeing when listing the contents of a directory. The h towards the end is especially helpful as it prints out the size of each file in a human readable format.

share|improve this answer
add comment

Often faster than find, if your locate database is up to date:

# locate '' | sed 's|/[^/]*$|/|g' | sort | uniq -c | sort -n | tee filesperdirectory.txt | tail

This dumps the entire locate database, strips off everything past the last '/' in the path, then the sort and "uniq -c" get you the number of files/directories per directory. "sort -n" piped to tail to get you the ten directories with the most things in them.

share|improve this answer
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.