Why is CPU frequency lower under load than when idle?

Question

I have a new Ubuntu 18.04 server on i9-9900KF CPU that I'd like to do some heavy calculations on. Since it has 16 threads, I changed all 16 /sys/devices/system/cpu/cpu*/cpufreq/scaling_governor to performance mode and stopped ondemand service with sudo systemctl disable ondemand so that it would get stuck after reboot (it does).

The result is puzzling. Before the change (under powersave governor), the frequencies are at 800MHz when idle (as expected by scaling_min_freq value) and 4700 MHz under load (scaling_max_freq value is 5000000).

After the change, the idle frequency is at 5000MHz as expected, but under load, it's again 4700MHz...? So under load it does not matter what the governor setting is, but with performance governor the idle frequency is even higher.

What could be causing this? My power unit should be good for 600W and I have nothing but mother board, dual fan water cooler, M2 SSD and 4-fan case (so I do not think power should be an issue here...?).

Answer 1

Typically, the maximum CPU clock frequency is a function of how many cores are active.

Use turbostat (linux-tools-common package) to read and decode the MSR_TURBO_RATIO_LIMIT register. Example (edited, 4 core processor):

$ sudo turbostat
...
cpu3: MSR_TURBO_RATIO_LIMIT: 0x23242526
35 * 100.0 = 3500.0 MHz max turbo 4 active cores
36 * 100.0 = 3600.0 MHz max turbo 3 active cores
37 * 100.0 = 3700.0 MHz max turbo 2 active cores
38 * 100.0 = 3800.0 MHz max turbo 1 active cores
...

Note: your processor is fairly new, I do not know if the distro version of turbostat is aware it. The upstream version is, I'm pretty sure.

Based on the comments, temperature might now be a concern. To observe, try this:

doug@s15:~$ sudo turbostat --quiet --Summary --show Busy%,Bzy_MHz,PkgTmp,PkgWatt,GFXWatt,IRQ --interval 15
Busy%   Bzy_MHz IRQ     PkgTmp  PkgWatt GFXWatt
0.03    1600    767     26      3.70    0.12
0.04    1600    778     26      3.70    0.12
0.06    1627    1138    25      3.71    0.12
37.28   3495    45585   47      23.90   0.12  <<< Load applied
100.00  3500    120635  53      58.51   0.12  <<< Observe CPU freq pinned at 4 core limit
100.00  3500    120604  57      59.28   0.12  <<< Observe processor package temperature rising
100.00  3500    120598  62      59.96   0.12
100.00  3500    120646  65      60.96   0.12
100.00  3500    120603  69      61.53   0.12
100.00  3500    120603  70      61.88   0.12
100.00  3500    120643  71      62.17   0.12
100.00  3500    120634  71      62.35   0.12
100.00  3500    120615  72      62.52   0.12
100.00  3500    120683  73      62.66   0.12
100.00  3500    120656  74      62.86   0.12
100.00  3500    120630  75      62.97   0.12
100.00  3500    120623  75      63.04   0.12
100.00  3500    120616  76      63.14   0.12

Answer 2

I would expect this is due to temperature, this is called throttling and it is done by the system (usually the OS or BIOS) to prevent harm from happening to the CPU. To tell if this is due to throttling or not, you would need to look (or ideally graph) at CPU temperature and CPU Frequency, if you found that the frequency decreases with higher temperature then it is throttling.

Dynamic frequency scaling (also known as CPU throttling) is a technique in computer architecture whereby the frequency of a microprocessor can be automatically adjusted "on the fly" depending on the actual needs, to conserve power and reduce the amount of heat generated by the chip

See also: https://en.wikipedia.org/wiki/Dynamic_frequency_scaling