Solution for my GPU question: I contacted Nvidia directly via live-chat, because I wanted to know how to fix the issue above in the most safest and easy way. There are a lot of old guides on the net. Some work, some don't work. I need the risk to be minimal, since I am on a stable LTS machine here.
This is the workaround Nvidia provided me for Ubuntu 12.04.4 LTS 64-bit:
It is expected that powermizer settings are not saved after reboot.
As a workaround, you can set "prefer maximum performance" on the
command line with
nvidia-settings -a [gpu:0]/GpuPowerMizerMode=1.
Replace gpu:0 with gpu:N if there is more than one GPU.
You can place that nvidia-settings invocation in ~/.xinitrc or some
other startup script. (It must be done after X is started)
I recommend doing it via the "start-up programs" utility, because it's relatively easy and because it ships with Ubuntu 12.04 LTS. Copy and paste the command into it, save and it will load on the next restart.
1. Open the start-up program utility and add the recommended command and a comment into it:
2. Activate it afterwards:
3. Restart your computer and check PowerMizer via nvidia-settings:
Done. Maximum power! Btw, this also helps if Unity is running slow or if you have any form stuttering, tearing, slowdown or lag in apps when PowerMizer switched from a low to a high performance-level. Credit goes to Mike from Nvidia customer care. He responded to me within 24 hours after asking my question, which is great. I am posting this information to let other Ubuntu users know how the issue was fixed. Thanks!
Update for Maxwell GTX 900-series GPUs: See This spreadsheet. It details setting up overclocking. Search for Nvidia to get the necessary details. This was tested on a GTX 970 on Kubuntu 14.10.