I have a web-site that I'm checking for performance issues and bugs, and i came across a caching code that caches thousands of files into one single directory.

I understand that this is not good and that I/O will degrade and i also heard about potential inode problem.

And i know how to fix the caching code, but the thing is that at this point fixing would be very expensive.

The question: What is the worst case scenario if i live it like it is right now? How will happen to the website? (right now this one single cache directory has 400K files)

I'm new to Ubuntu. And i understand that this is might be an off-topic. But i think this is a "system" question and it does not belong to the 'programming' part of the stackoverflow.


UPDATE: The file system is UFS

  • I have. I think i will post this on webmasters as well. Thank you :) – rinchik Jan 31 '13 at 15:33
  • 1
    Important: what is the filesystem ext3? ext4? fat? :) – Rinzwind Jan 31 '13 at 15:37
  • The files system is ufs :) Will update my post shortly – rinchik Jan 31 '13 at 15:50

UFS works with inodes so the maximum number of files depends on the inode to space ratio that has been set. And that is a lot... a lot more than 400k.

2 problems:

  • Creating new files will take more and more time.

  • You will also run into other problems: ls will error out with Argument list too long. since it will expand all the files (ls -l * turns into ls -l file1 file2 file3 file4 etc etc. Same goes for rm and some other commands.

I know you will not like it but ... it IS best to use sub directories. Yes, can be a hell of a lot of work but you are bound to end up having to do it anyways. Better be prepared now while you have the time than after the server crashed.

Alternative: - analyse if you can drop some files and not cache them.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.