How to change frequency range for intel_pspace performance govenor

Question

I want to set higher CPU frequency on Ubuntu 20.04 based system (KDE neon) with 5.4.0-58-generic kernel. The CPU is Intel(R) Core(TM) i7-8700, with the hardware limits 800 MHz - 4.60 GHz.

Setting

sudo cpupower frequency-set --min 4500Hz 

as suggested here does not work. Frequency still ranges within 800 MHz and 4.60 GHz.

Example output of cpufreq-info is:

analyzing CPU 11:
  driver: intel_pstate
  CPUs which run at the same hardware frequency: 11
  CPUs which need to have their frequency coordinated by software: 11
  maximum transition latency: 4294.55 ms.
  hardware limits: 800 MHz - 4.60 GHz
  available cpufreq governors: performance, powersave
  current policy: frequency should be within 800 MHz and 4.60 GHz.
                  The governor "performance" may decide which speed to use
                  within this range.
  current CPU frequency is 4.21 GHz.

for all cores.

Changing scaling_min_freq does not help either:

MAX_CPU=$((`nproc --all` - 1))
for i in $(seq 0 $MAX_CPU); do
    echo 4500000 | sudo tee /sys/devices/system/cpu/cpu$i/cpufreq/scaling_min_freq
done

Example output of cpufreq-info

analyzing CPU 11:
  driver: intel_pstate
  CPUs which run at the same hardware frequency: 11
  CPUs which need to have their frequency coordinated by software: 11
  maximum transition latency: 4294.55 ms.
  hardware limits: 800 MHz - 4.60 GHz
  available cpufreq governors: performance, powersave
  current policy: frequency should be within 4.50 GHz and 4.60 GHz.
                  The governor "performance" may decide which speed to use
                  within this range.
  current CPU frequency is 4.23 GHz.

Even though frequency should be within 4.50 GHz and 4.60 GHz, current CPU frequency stays 4.23 GHz. It seems that scaling_min_freq is only used for printing information and does not really change performance governor.

I also tried pstate-frequency:

sudo pstate-frequency -S -p max 
pstate-frequency version 3.11.0
    pstate::CPU_DRIVER   -> intel_pstate
    pstate::CPU_GOVERNOR -> performance
    pstate::TURBO        -> 0 [ON]
    pstate::CPU_MIN      -> 99% [4554000KHz]
    pstate::CPU_MAX      -> 100% [4600000KHz]

But this again only changes scaling_min_freq and the output of cpufreq-info remains ambiguous

analyzing CPU 11:
  driver: intel_pstate
  CPUs which run at the same hardware frequency: 11
  CPUs which need to have their frequency coordinated by software: 11
  maximum transition latency: 4294.55 ms.
  hardware limits: 800 MHz - 4.60 GHz
  available cpufreq governors: performance, powersave
  current policy: frequency should be within 4.55 GHz and 4.60 GHz.
                  The governor "performance" may decide which speed to use
                  within this range.
  current CPU frequency is 4.24 GHz.

Apart from using legacy cpufreq_userspace driver instead of intel_pstate, is there any way to set higher minimum (or better constant) CPU frequency?

Answer 1

The processor can back off the CPU frequencies by itself, even in performance mode. In performance mode, with HWP enabled, you should not be able to detect any performance degradation. Note also that adjusting various minimums would not have any effect on performance mode.

Now, if the desire is really to always get an exact CPU reported frequency then here are some methods:

first you have to deal with maximum CPU frequency as a function of active cores. Turbostat will list this information for you, if you do not ask it to be quiet. Example 1 (i7-2600K):

$ sudo turbostat
...
cpu5: MSR_TURBO_RATIO_LIMIT: 0x23242526
35 * 100.0 = 3500.0 MHz max turbo 4 active cores
36 * 100.0 = 3600.0 MHz max turbo 3 active cores
37 * 100.0 = 3700.0 MHz max turbo 2 active cores
38 * 100.0 = 3800.0 MHz max turbo 1 active cores

So even if I set the minimum CPU frequency to 3.8 GHz, the processor itself will override that as a function of active cores. Therefore I would have to set the maximum CPU frequency to 3.5 GHz to avoid this issue.

Example 2 (i5-9600K):

cpu5: MSR_TURBO_RATIO_LIMIT: 0x2b2b2e2e2e2e2e2e
43 * 100.0 = 4300.0 MHz max turbo 8 active cores << I only have 6 cores
43 * 100.0 = 4300.0 MHz max turbo 7 active cores << I only have 6 cores
46 * 100.0 = 4600.0 MHz max turbo 6 active cores
46 * 100.0 = 4600.0 MHz max turbo 5 active cores
46 * 100.0 = 4600.0 MHz max turbo 4 active cores
46 * 100.0 = 4600.0 MHz max turbo 3 active cores
46 * 100.0 = 4600.0 MHz max turbo 2 active cores
46 * 100.0 = 4600.0 MHz max turbo 1 active cores

Observe that I have it set not to do CPU frequency reduction as a function of active cores.

Disclaimer: do not do this unless you know for certain that your system is capable of dealing with the heat, and current requirements. Also requires an unlocked processor.

Now, even so, the CPU frequency might not hold at 4.6 GHz in performance mode:

$ sudo turbostat --Summary --quiet --show Busy%,Bzy_MHz,IRQ,PkgWatt,PkgTmp --interval 10
Busy%   Bzy_MHz IRQ     PkgTmp  PkgWatt
0.14    4579    650     32      2.10  <<<
0.40    4600    507     32      2.27
0.09    4600    270     31      2.01
0.13    4600    322     32      2.10
0.02    4600    205     31      1.99
0.11    4600    229     32      2.04
0.10    4600    269     32      2.02
0.08    4600    300     32      2.00
0.04    4598    443     32      2.01
0.05    4600    215     32      1.98
0.02    4586    210     32      1.95 <<
0.05    4594    233     32      1.97 <<
0.01    4600    246     32      1.95

This can be eliminated by disabling idle states entirely, at the cost of a lot of power:

$ echo 1 | sudo tee /sys/devices/system/cpu/cpu*/cpuidle/state*/disable

.

$ sudo turbostat --Summary --quiet --show Busy%,Bzy_MHz,IRQ,PkgWatt,PkgTmp --interval 10
Busy%   Bzy_MHz IRQ     PkgTmp  PkgWatt
100.26  4600    60109   47      52.88  <<< not 100% busy, but 100% idle state 0
100.26  4600    60113   48      53.01  <<< if no idle states enabled the kernel will use 0.
100.26  4600    60107   48      53.01
100.26  4600    60183   48      53.01

O.K. but there is also another method. it turns out that Idle State 2 is a problem with HWP, so just disable it instead:

doug@s18:~/idle$ echo 1 | sudo tee /sys/devices/system/cpu/cpu*/cpuidle/state*/disable
1
doug@s18:~/idle$ sudo ~/c/msr-decoder^C
doug@s18:~/idle$ echo 0 | sudo tee /sys/devices/system/cpu/cpu*/cpuidle/state*/disable
0
doug@s18:~/idle$ echo 1 | sudo tee /sys/devices/system/cpu/cpu*/cpuidle/state2/disable
1
doug@s18:~/idle$ grep . /sys/devices/system/cpu/cpu3/cpuidle/state*/disable
/sys/devices/system/cpu/cpu3/cpuidle/state0/disable:0
/sys/devices/system/cpu/cpu3/cpuidle/state1/disable:0
/sys/devices/system/cpu/cpu3/cpuidle/state2/disable:1
/sys/devices/system/cpu/cpu3/cpuidle/state3/disable:0
/sys/devices/system/cpu/cpu3/cpuidle/state4/disable:0
/sys/devices/system/cpu/cpu3/cpuidle/state5/disable:0
/sys/devices/system/cpu/cpu3/cpuidle/state6/disable:0
/sys/devices/system/cpu/cpu3/cpuidle/state7/disable:0
/sys/devices/system/cpu/cpu3/cpuidle/state8/disable:0

.

$ sudo turbostat --Summary --quiet --show Busy%,Bzy_MHz,IRQ,PkgWatt,PkgTmp --interval 10
Busy%   Bzy_MHz IRQ     PkgTmp  PkgWatt
0.02    4600    332     35      1.96
0.03    4600    209     35      1.94
0.09    4600    290     35      2.00
0.17    4600    377     35      2.07
0.03    4600    205     34      1.94
0.05    4600    192     35      1.95
0.15    4600    274     35      2.04
0.04    4600    208     35      1.96
0.11    4600    279     35      2.02
0.06    4600    219     34      1.97
0.13    4600    251     34      2.03
0.03    4600    173     34      1.94
0.11    4600    343     34      2.02
0.02    4600    205     33      1.94
0.03    4600    180     34      1.94
0.04    4600    183     34      1.95
0.12    4600    250     34      2.03
0.08    4600    236     33      1.99
0.02    4600    225     34      1.94
0.02    4600    165     34      1.93
0.01    4600    173     33      1.93
0.01    4600    170     33      1.93
0.02    4600    200     33      1.95
0.02    4600    199     33      1.94
0.02    4600    218     33      1.94
0.01    4600    161     33      1.93
0.01    4600    170     33      1.94
0.01    4600    187     33      1.94
0.02    4600    218     33      1.95
0.02    4600    176     33      1.94
0.01    4600    212     32      1.94

For completeness, do the i7-2600K, with no HWP.

$ grep . /sys/devices/system/cpu/cpu3/cpufreq/*
/sys/devices/system/cpu/cpu3/cpufreq/affected_cpus:3
/sys/devices/system/cpu/cpu3/cpufreq/cpuinfo_max_freq:3800000
/sys/devices/system/cpu/cpu3/cpufreq/cpuinfo_min_freq:1600000
/sys/devices/system/cpu/cpu3/cpufreq/cpuinfo_transition_latency:20000
/sys/devices/system/cpu/cpu3/cpufreq/related_cpus:3
/sys/devices/system/cpu/cpu3/cpufreq/scaling_available_governors:ondemand performance schedutil
/sys/devices/system/cpu/cpu3/cpufreq/scaling_cur_freq:1605486
/sys/devices/system/cpu/cpu3/cpufreq/scaling_driver:intel_cpufreq
/sys/devices/system/cpu/cpu3/cpufreq/scaling_governor:ondemand
/sys/devices/system/cpu/cpu3/cpufreq/scaling_max_freq:3800000
/sys/devices/system/cpu/cpu3/cpufreq/scaling_min_freq:1600000
/sys/devices/system/cpu/cpu3/cpufreq/scaling_setspeed:<unsupported>
doug@s15:~/diag$ grep . /sys/devices/system/cpu/cpu3/cpufreq/*^C
doug@s15:~/diag$ echo performance | sudo tee /sys/devices/system/cpu/cpu*/cpufreq/scaling_governor
performance
doug@s15:~/diag$ echo 3500000 | sudo tee /sys/devices/system/cpu/cpu*/cpufreq/scaling_max_freq
3500000
doug@s15:~/diag$ sudo turbostat --Summary --quiet --show Busy%,Bzy_MHz,IRQ,PkgWatt,PkgTmp --interval 10
Busy%   Bzy_MHz IRQ     PkgTmp  PkgWatt
0.02    1806    453     28      3.75
0.02    1889    340     28      3.73
0.01    1896    268     31      3.72
0.02    1771    437     28      3.73

Hey, why is the CPU frequency not stable? Well, for non HWP processors, only the disable all idle states method works. A bit of a response boost can be obtained via setting energy performance bias (only used in non HWP mode):

$ cat /sys/devices/system/cpu/cpu3/power/energy_perf_bias
6
doug@s15:~/diag$ echo 0 | sudo tee /sys/devices/system/cpu/cpu3/power/energy_perf_bias
0
$ echo 1 | sudo tee /sys/devices/system/cpu/cpu*/cpuidle/state*/disable

.

$ sudo turbostat --Summary --quiet --show Busy%,Bzy_MHz,IRQ,PkgWatt,PkgTmp --interval 10
Busy%   Bzy_MHz IRQ     PkgTmp  PkgWatt
100.00  3500    20114   52      49.06
100.00  3500    20146   54      49.49
100.00  3500    20096   58      49.87
100.00  3500    20086   59      50.17
100.00  3500    22609   61      50.50
100.00  3500    20110   62      51.02
100.00  3500    20105   63      51.53
100.00  3500    20081   65      51.78
100.00  3500    20059   66      52.02
100.00  3500    20090   67      52.23
100.00  3500    20717   68      52.37
100.00  3500    20066   68      52.48
100.00  3500    20074   68      52.58
100.00  3500    20222   71      52.78
100.00  3500    20109   70      52.83
100.00  3500    20075   70      52.81
100.00  3500    21040   70      52.92
100.00  3500    20066   70      52.95