There are three major performance parameters governing kernel optimization, one of which rarely gets discussed in context of the other two and almost never gets benchmarked. This has reduced and skewed our perception of performance:
- Throughput: What gets benchmarked and discussed by the media. A measure of how much data can make it through the processor in a given timeframe.
- Energy efficiency: What also often gets benchmarked; depending on how you measure it, how costly (in energy or heat) it is to process a certain amount of data or to keep a system running over a given period of time. This has implications for both battery life in laptops and the cost of running servers or other "always on" hardware.
- Latency: Almost never gets benchmarked, but just as important. It is a measure of the average amount of time it takes for a signal or data to travel through a path, usually measured in milliseconds. The word "jitter" describes variations in latency over time, and is a sub-component of latency performance.
This is a classic case of "here's 3 options, pick any two." You can have throughput and energy efficiency, but will sacrifice latency. You can have throughput and low latency, but will sacrifice energy efficiency. You can have low latency and energy efficiency, but that will sacrifice throughput.
One example of this tradeoff is race-to-sleep: https://en.wikipedia.org/wiki/Race_to_sleep. Unfortunately, it still takes CPUs several ms to "wake up" from various levels of sleep states (the deeper the sleep, the longer it takes to wake up), which makes CPU frequency scaling horrible for latency performance and probably one of the single biggest offenders. This is why even Mac OS systems benefit immensely from disabling CPU frequency scaling to prevent buffer overruns and underruns (a buffer overrun is when a process takes too long to finish and reroute, and so the last bit gets discarded as the buffer gets refilled; whereas an underrun is a failure to sufficiently fill the buffer in time for processing. both are commonly called "xruns" and indicate potential data loss)
Another is logical cores and symmetric multi-threading, which is a subset of "race to sleep" by using all cores more efficiently, but at the cost of increasing the time it takes for any individual process to complete. This is also good for energy efficiency and throughput, but not latency, which is concerned with how reliably and quickly individual "mission critical" tasks can complete, not a sum total group of tasks.
-generic: for any use case that does not deal with latency and the guaranteed routing of a certain amount of information into the processor and out to its destination. Generic in general provides greatest throughput performance as well as energy efficiency, but deprioritizes latency
the -lowlatency, -rt and -realtime kernels provided varying shades of increased attention to latency, with increasing sacrifice or deprioritization of throughput and/or energy efficiency. Use cases determine which are most appropriate for which circumstances, and these aren't the only choices. For example, there is also https://liquorix.net/ which claims to be optimized for most common usage scenarios by making small sacrifices in throughput for relatively large gains in latency performance.
There are components of this question that confusingly overlap with other questions, and parts of this question are becoming obsolete as the performance distinctions between kernel lines are disappearing. For example, the -generic kernel has included many low latency optimizations (such as PREEMPT) that make the other specialized kernels even more specialized. It's hard to give specific numbers, but on my system, -generic can now handle latencies down to about 20ms with some reliability (minimal or no buffer overruns or underruns).
I honestly think this fragmentation exists from
- a byproduct of open source software, where you get forking of kernel lines and software generally to optimize them for specific use cases (yes, "-generic" is a specific use case, just one that specifically happens to cover most use cases :) and
- Almost completely ignoring the importance of low latency performance tuning to the -generic user's experience in both Linux and Windows paradigms.
Mac OS X is different, and you can read why here: https://www.cse.unsw.edu.au/~cs9242/10/lectures/09-OSXAudiox4.pdf (it was political and economic, not genius and vision!). The result was a slight tradeoff in throughput performance for a system that handles latency issues better. And thus Mac OS X -- and its iOS derivative -- ended up cornering the market on digital multimedia production. With one kernel. And non-multimedia users don't complain about Mac OS X's consistently lower throughput performance, because no one actually cares about 29sec vs 30sec to transcode a file, or 4.75sec vs 5sec to start a program (even the benchmarkers and "performance freaks" don't notice these sorts of discrepancies on a daily, real-world basis), and everyone cares about whether the UX stutters and audio glitches out. It's a psychological thing related to the importance of latency and responsiveness, not a throughput performance thing (which is the only thing the benchmark wars consider), and up to this point only Mac OS X really gave it serious consideration on a whole OS design.
https://liquorix.net/ may be more in line with Mac's kernel in terms of its tuned performance priorities (I am confirming that now). And that is the direction the -generic kernel is headed, I believe (and hope). Which will be good enough for 90-some percent of users, except those on either extreme who TRULY need a. All the throughput they can get or b. extremely low glitch-free latency and jitter. And for those cases, there will always be some sort of custom kernel or kernels. I believe NASA rolls their own, but I could be wrong :)