15

I do not have much knowledge about operating system internal workings, so why do we have limits for maximum number of open files and running processes in Linux?

I would appreciate it if anyone could help me understand.

4 Answers 4

17

This is mainly for historical reasons. On older Linux mainframes, many users would connect and use the mainframe's resources. Of course, itwas necessary to limit, and since such operations like file handles and processes were built into the kernel, it was limited there. It also helps limit attacks like the fork bomb. A defense against the form bomb using process limits is shown here.

It also helps keep complex services and daemons in check by not allowing runaway forking and file opening, similar to what the fork bomb tries to do.

Also to be noted are things like the amount of RAM and CPU available, and the fact that a 32-bit counter can only reference so much(only 4294967296 entries that can be counted by a 32-bit counter), but such limits are far above those usually set by programmers and system administrators. Anyway, long before you have 4294967296 processes, your machine would have probably been rebooted, either just as planned, or as it began to lock up due to starving another resource.

Unless you run the Titan with its 584 TiB of memory(you won't, since Linux cannot be run as one instance on a supercomputer), you probably won't reach this process limit anytime soon. Even then, the average process would only have roughly 146KB of memory, assuming no shared memory.

1
  • 2
    +1 for fork bomb, showed how dangerous these recursive methods can be if used in shell without caution! Nov 11, 2012 at 19:37
9

There are limits to most everything, both in life and in Linux. Sometimes the limits are very high and not worth worrying about, and sometimes they are too low, set arbitrarily by a lazy programmer, or set by limits to the hardware.

Arbitrary limits come about when a programmer makes a decision such as the maximum number of characters allowed in a field, such as an address. It's a lot easier to set some arbitrary limit than to dynamically allocate memory as needed. You don't want to allocate all available memory to a simple input field, but you don't want to allow the input to overwrite other memory that may be important, either. For open files and processes, there may be an arbitrary limit, or it may be limited by available memory. If the latter, then bad things will start to happen as you approach the limit, and the system may hang, so it's often good to have a limit that comes into play before that happens. Sometimes, the arbitrary limits may be determined at startup depending on memory or disk space, and are usually fairly intelligent, but still arbitrary - someone decided where performance would become intolerable or dangerous, and set limits to avoid that situation.

Then there are limits set by hardware, such as the size of an integer or character. If you have a 32-bit system, then there are limits set by the maximum size of an integer (4,294,967,295 if unsigned, or half that if signed).

2
  • 1
    Limits which appear arbitrary may actually be standards based. International standard limit postal address lines to 39 characters. Such a limit may appear arbitrary. A limit of 60 characters for an address would likely be arbitrary. Unfortunately, too few programmers check the standards so many (sometimes correct) limits are arbitrarily decided.
    – BillThor
    Nov 12, 2012 at 3:16
  • I agree; when I was still working as a programmer, I'd spend a lot of time looking for stated limits, either as you said, or defined in the OS as a macro (such as max_path, etc). But I'll have to say that arbitrary limits are usually better than no limits at all, especially for field lengths. :) Nov 12, 2012 at 22:12
7

For processes, /proc/sys/kernel/pid_max is the maximum pid allowed, so this is the hard limit of the number of processes that can run at any instant. However, limits on memory normally come into play well before that.

For the upper limit on open files on the entire machine, /proc/sys/fs/file-max is the hard limit; this is based on the amount of memory in the machine, so it will vary. The kernel sets this to be:

    n = (mempages * (PAGE_SIZE / 1024)) / 10;
    files_stat.max_files = max_t(unsigned long, n, NR_FILE);

which works out to be about 100 for each 1 MB of RAM.

Per process limits are different. ulimit will define these.

-5

We have a limit because computers have limits , I have 4GB of RAM , So I can't run 15 games now can I?

I most likely can run it with some lag but it wouldn't be human , It is the limit on your PC and Linux's Kernal , You would need a better kernal and alot of RAM to be able to hold any program that is thrown at you.

Like my laptop when I ran Windows 7 , I could Play Battlefield , But It took 80% of my RAM so if I tried to play "Need For Speed" , That takes 60% Ussually , I can open both but then my whole system is slowed down since I am running 2 Heavy Duty Programs , They are both fighting for RAM (Random Access Memory)

In short words , A normal computer can't run alot of programs at once Music , Notepad , Browser , Email , Game , You think that is alot on a computer but that is mere , A game like World of Warcraft takes alot more RAM and makes your computer struggle (Unless you have 8-16 GB of RAM like expensive laptops do!)

8
  • Thanks for the information about processes as we know there will be high CPU load if many process will run simultaneously, and the traffic on CPU cores will be so high to handle it and with time your system will go in hang state, not giving any proper time slice to any of the process to execute i guess if i am not wrong. But why there is also limit for open files in Linux? Nov 11, 2012 at 17:46
  • I do not know about a limit of open files , I have not experienced that at all , Maybe it is your computers limited compared to mine , I am unsure at that matter. I always have a few 10 files open , Then a web browser , Music Player , and the Terminal open. Nov 11, 2012 at 17:53
  • It generally happens with me when my mysql database gives me following "120428 20:30:01 [ERROR] /usr/sbin/mysqld: Can't open file: './djlabo_open/cart_layout.frm' (errno: 24)" error in its error log because linux is not letting it to open this database file. Nov 11, 2012 at 18:07
  • That's probably something waaaay different then an OS limit to open files.
    – Nanne
    Nov 11, 2012 at 18:40
  • 1
    @Terzuz It really is based on whether people consider each others' answers factually correct. Many people(including me, unfortunately) feel as if this answer may not be really addressing the limits mentioned in the question, but other limits.
    – nanofarad
    Nov 11, 2012 at 21:03

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .