How to install Nvidia drivers on a server

Question

I am trying to install nvidia drivers instead of nouveau on a remote server running Ubuntu 18.04, which I am accessing via remote desktop (running Plasma KDE). I have followed the instructions from other posts trying to install the proprietary drivers, then purging and reinstalling multiple times. I am not sure the driver is installed correctly.

What I did (following this post Install Nvidia Drivers on Ubuntu Server 18.04):

sudo apt-get purge nvidia*
sudo apt-get autoremove
sudo add-apt-repository ppa:graphics-drivers
sudo apt-get install nvidia-driver-430
sudo reboot

This seems to go smoothly.

I then check nvidia-smi:

+-----------------------------------------------------------------------------+
| NVIDIA-SMI 430.40       Driver Version: 430.40       CUDA Version: 10.1     |
|-------------------------------+----------------------+----------------------+
| GPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |
| Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |
|===============================+======================+======================|
|   0  GeForce GTX 1080    Off  | 00000000:01:00.0 Off |                  N/A |
| 33%   37C    P8     7W / 200W |     54MiB /  8119MiB |      0%      Default |
+-------------------------------+----------------------+----------------------+
                                                                           
+-----------------------------------------------------------------------------+
| Processes:                                                       GPU Memory |
|  GPU       PID   Type   Process name                             Usage      |
|=============================================================================|
|    0      1273      G   /usr/lib/xorg/Xorg                             9MiB |
|    0      1339      G   /usr/bin/sddm-greeter                         42MiB |
+-----------------------------------------------------------------------------+

This makes me think that the driver has been installed correctly, however, sudo nvidia-settings fails to run:

ERROR: Unable to load info from any available system                                                                                                                               
                                                                                                                                                                               
                                                                                                                                                                               
(nvidia-settings:2278): GLib-GObject-CRITICAL **: 13:28:34.205: g_object_unref: assertion 'G_IS_OBJECT (object)' failed                                                            
** Message: 13:28:34.206: PRIME: No offloading required. Abort
** Message: 13:28:34.206: PRIME: is it supported? no

Also, glxinfo | grep nvidia returns nothing.

The last two facts make me think that the driver has not been installed correctly.

Finally, I'm not sure what this is saying exactly, but here's output of dmesg | grep nvidia:

[    1.257269] nvidia: loading out-of-tree module taints kernel.
[    1.257467] nvidia: module license 'NVIDIA' taints kernel.
[    1.261066] nvidia: module verification failed: signature and/or required key missing - tainting kernel
[    1.267769] nvidia-nvlink: Nvlink Core is being initialized, major device number 240
[    1.268292] nvidia 0000:01:00.0: vgaarb: changed VGA decodes: olddecodes=io+mem,decodes=none:owns=io+mem
[    1.389997] nvidia-modeset: Loading NVIDIA Kernel Mode Setting Driver for UNIX platforms  430.40  Sun Jul 21 04:57:42 CDT 2019
[    1.390918] [drm] [nvidia-drm] [GPU ID 0x00000100] Loading driver
[    2.091441] [drm] Initialized nvidia-drm 0.0.0 20160202 for 0000:01:00.0 on minor 0
[    4.258679] nvidia-uvm: Loaded the UVM driver in 8 mode, major device number 238

Where I think the issue might be:

In some posts such as:

How can I install Nvidia drivers on Ubuntu 18.04 with secure boot?

Unable to install Nvidia drivers on Ubuntu 18.04.1

I found suggestions that during the installation, you are asked a password, which you then need to type again on reboot. I did not have this step. In fact, I am unsure how to tackle any suggestions on action being taken before the computer boots up (such as entering BIOS or disabling Secure Boot because as I said, it's a remote server which I access either via ssh or remote desktop.

Extra information:

Running glxinfo | grep OpenGL:

OpenGL vendor string: VMware, Inc.
OpenGL renderer string: llvmpipe (LLVM 8.0, 256 bits)
OpenGL core profile version string: 3.3 (Core Profile) Mesa 19.0.2
OpenGL core profile shading language version string: 3.30
OpenGL core profile context flags: (none)
OpenGL core profile profile mask: core profile
OpenGL core profile extensions:
OpenGL version string: 3.1 Mesa 19.0.2
OpenGL shading language version string: 1.40
OpenGL context flags: (none)
OpenGL extensions:
OpenGL ES profile version string: OpenGL ES 3.0 Mesa 19.0.2
OpenGL ES profile shading language version string: OpenGL ES GLSL ES 3.00
OpenGL ES profile extensions:

Running sudo nvidia-settings:

Error code listed above, but a small blank window appears.

/etc/X11/Xorg.conf file contents:

# nvidia-xconfig: X configuration file generated by nvidia-xconfig
# nvidia-xconfig:  version 430.40

Section "ServerLayout"
    Identifier     "Layout0"
    Screen      0  "Screen0"
    InputDevice    "Keyboard0" "CoreKeyboard"
    InputDevice    "Mouse0" "CorePointer"
EndSection

Section "Files"
EndSection

Section "InputDevice"
    # generated from default
    Identifier     "Mouse0"
    Driver         "mouse"
    Option         "Protocol" "auto"
    Option         "Device" "/dev/psaux"
    Option         "Emulate3Buttons" "no"
    Option         "ZAxisMapping" "4 5"
EndSection

Section "InputDevice"
    # generated from default
    Identifier     "Keyboard0"
    Driver         "kbd"
EndSection

Section "Monitor"
    Identifier     "Monitor0"
    VendorName     "Unknown"
    ModelName      "Unknown"
    Option         "DPMS"
EndSection

Section "Device"
    Identifier     "Device0"
    Driver         "nvidia"
    VendorName     "NVIDIA Corporation"
EndSection

Section "Screen"
    Identifier     "Screen0"
    Device         "Device0"
    Monitor        "Monitor0"
    DefaultDepth    24
    SubSection     "Display"
        Depth       24
    EndSubSection
EndSection

Terminal output when running sudo apt-get install nvidia-driver-430

Reading package lists... Done
Building dependency tree       
Reading state information... Done
The following additional packages will be installed:
  libxnvctrl0 nvidia-compute-utils-430 nvidia-dkms-430 nvidia-kernel-common-430 nvidia-kernel-source-430
  nvidia-prime nvidia-settings nvidia-utils-430 screen-resolution-extra
The following NEW packages will be installed:
  libxnvctrl0 nvidia-compute-utils-430 nvidia-dkms-430 nvidia-driver-430 nvidia-kernel-common-430
  nvidia-kernel-source-430 nvidia-prime nvidia-settings nvidia-utils-430 screen-resolution-extra
0 upgraded, 10 newly installed, 0 to remove and 162 not upgraded.
Need to get 0 B/13.8 MB of archives.
After this operation, 39.0 MB of additional disk space will be used.
Do you want to continue? [Y/n] y
Selecting previously unselected package libxnvctrl0:amd64.
(Reading database ... 250513 files and directories currently installed.)
Preparing to unpack .../0-libxnvctrl0_418.56-0ubuntu0~gpu18.04.1_amd64.deb ...
Unpacking libxnvctrl0:amd64 (418.56-0ubuntu0~gpu18.04.1) ...
Selecting previously unselected package nvidia-compute-utils-430.
Preparing to unpack .../1-nvidia-compute-utils-430_430.40-0ubuntu0~gpu18.04.1_amd64.deb ...
Unpacking nvidia-compute-utils-430 (430.40-0ubuntu0~gpu18.04.1) ...
Selecting previously unselected package nvidia-kernel-source-430.
Preparing to unpack .../2-nvidia-kernel-source-430_430.40-0ubuntu0~gpu18.04.1_amd64.deb ...
Unpacking nvidia-kernel-source-430 (430.40-0ubuntu0~gpu18.04.1) ...
Selecting previously unselected package nvidia-kernel-common-430.
Preparing to unpack .../3-nvidia-kernel-common-430_430.40-0ubuntu0~gpu18.04.1_amd64.deb ...
Unpacking nvidia-kernel-common-430 (430.40-0ubuntu0~gpu18.04.1) ...
Selecting previously unselected package nvidia-dkms-430.
Preparing to unpack .../4-nvidia-dkms-430_430.40-0ubuntu0~gpu18.04.1_amd64.deb ...
Unpacking nvidia-dkms-430 (430.40-0ubuntu0~gpu18.04.1) ...
Selecting previously unselected package nvidia-utils-430.
Preparing to unpack .../5-nvidia-utils-430_430.40-0ubuntu0~gpu18.04.1_amd64.deb ...
Unpacking nvidia-utils-430 (430.40-0ubuntu0~gpu18.04.1) ...
Selecting previously unselected package nvidia-driver-430.
Preparing to unpack .../6-nvidia-driver-430_430.40-0ubuntu0~gpu18.04.1_amd64.deb ...
Unpacking nvidia-driver-430 (430.40-0ubuntu0~gpu18.04.1) ...
Selecting previously unselected package nvidia-prime.
Preparing to unpack .../7-nvidia-prime_0.8.8.2_all.deb ...
Unpacking nvidia-prime (0.8.8.2) ...
Selecting previously unselected package screen-resolution-extra.
Preparing to unpack .../8-screen-resolution-extra_0.17.3_all.deb ...
Unpacking screen-resolution-extra (0.17.3) ...
Selecting previously unselected package nvidia-settings.
Preparing to unpack .../9-nvidia-settings_418.56-0ubuntu0~gpu18.04.1_amd64.deb ...
Unpacking nvidia-settings (418.56-0ubuntu0~gpu18.04.1) ...
Setting up nvidia-prime (0.8.8.2) ...
Processing triggers for mime-support (3.60ubuntu1) ...
Setting up nvidia-utils-430 (430.40-0ubuntu0~gpu18.04.1) ...
Setting up nvidia-kernel-common-430 (430.40-0ubuntu0~gpu18.04.1) ...
update-initramfs: deferring update (trigger activated)
Setting up nvidia-compute-utils-430 (430.40-0ubuntu0~gpu18.04.1) ...
Warning: The home dir /nonexistent you specified can't be accessed: No such file or directory
Adding system user `nvidia-persistenced' (UID 127) ...
Adding new group `nvidia-persistenced' (GID 135) ...
Adding new user `nvidia-persistenced' (UID 127) with group `nvidia-persistenced' ...
Not creating home directory `/nonexistent'.
Processing triggers for libc-bin (2.27-3ubuntu1) ...
Setting up nvidia-kernel-source-430 (430.40-0ubuntu0~gpu18.04.1) ...
Setting up screen-resolution-extra (0.17.3) ...
Processing triggers for man-db (2.8.3-2ubuntu0.1) ...
Processing triggers for dbus (1.12.2-1ubuntu1.1) ...
Setting up libxnvctrl0:amd64 (418.56-0ubuntu0~gpu18.04.1) ...
Setting up nvidia-dkms-430 (430.40-0ubuntu0~gpu18.04.1) ...
update-initramfs: deferring update (trigger activated)
INFO:Enable nvidia
DEBUG:Parsing /usr/share/ubuntu-drivers-common/quirks/dell_latitude
DEBUG:Parsing /usr/share/ubuntu-drivers-common/quirks/lenovo_thinkpad
DEBUG:Parsing /usr/share/ubuntu-drivers-common/quirks/put_your_quirks_here
Loading new nvidia-430.40 DKMS files...
Building for 4.15.0-58-generic
Building for architecture x86_64
Building initial module for 4.15.0-58-generic
Done.

nvidia:
Running module version sanity check.
 - Original module
   - No original module exists within this kernel
 - Installation
   - Installing to /lib/modules/4.15.0-58-generic/updates/dkms/

nvidia-modeset.ko:
Running module version sanity check.
 - Original module
   - No original module exists within this kernel
 - Installation
   - Installing to /lib/modules/4.15.0-58-generic/updates/dkms/

nvidia-drm.ko:
Running module version sanity check.
 - Original module
   - No original module exists within this kernel
 - Installation
   - Installing to /lib/modules/4.15.0-58-generic/updates/dkms/

nvidia-uvm.ko:
Running module version sanity check.
 - Original module
   - No original module exists within this kernel
 - Installation
   - Installing to /lib/modules/4.15.0-58-generic/updates/dkms/

depmod...

DKMS: install completed.
Setting up nvidia-driver-430 (430.40-0ubuntu0~gpu18.04.1) ...
Setting up nvidia-settings (418.56-0ubuntu0~gpu18.04.1) ...
Processing triggers for initramfs-tools (0.130ubuntu3.8) ...
update-initramfs: Generating /boot/initrd.img-4.15.0-58-generic
I: The initramfs will attempt to resume from /dev/dm-1
I: (/dev/mapper/vg0-swap)
I: Set the RESUME variable to override this.
Processing triggers for libc-bin (2.27-3ubuntu1) ...

Answer 1

Recently I had an issue with my server with 4 A100 GPU cards. The driver was working normal but had to install a newer one to access newer Cuda ( updating from (450 to 510), here is what worked for me smoothly on ubuntu 20.04 but works on ubuntu 18.04 as well:

1. avoid using apt-get to install Nvidia driver preferably! For me, it was essential to avoid any x11 Xorg, So I avoided it. But, it might be ok in your case.

2. purge all (trust me, it feels risky but works):

sudo apt-get remove --purge '^nvidia-.*'
sudo apt-get remove --purge '^libnvidia-.*'
sudo apt-get remove --purge '^cuda-.*'
sudo nvidia-uninstaller

3. go here, it says CUDA toolkit, but it has both Nvidia-driver and Cuda together. Based on Table 3 select the desired version you want to install` Note: by Cuda it actually means max available Cuda. In other words, if you install Cuda toolkit 11.6 ( which has Nvidia driver v510; you can work with older Cuda (like 11.5...and older), without changing the driver version.

4. from here click on CUDA Toolkit XX.X.X (Month 20XX), and also open its Versioned Online Documentation.

4.1 install desired kernel and tools: from Versioned Online Documentation: go to Installation Guide Linux and take a look at table 1. Basically, it says required kernel versions for Nvidia-driver: in my case I was installing 11.6.1 on Ubuntu 20.04.3, So I installed the same version of the kernel image and its headers and then booted with that kernel ( check your kernel version with uname -r if it is the same just go to next step, otherwise: (replace $(name -r) with exact version mentioned in Table 1)

 sudo apt install linux-image-$(uname -r)

4.2 install kernel headers: #(replace $(uname -r) with exact version mentioned in Table 1)

sudo apt install linux-headers-$(uname -r) 

4.3 install desired driver+cuda ( it's called CUDA Toolkit): follow the first link CUDA Toolkit XX.X.X and click on Linux>X86_64>Ubuntu> your ubuntu version > run file (local) ( all are fine I preferred this)

then use two provided commands in the terminal and the rest is easy peasy.

Happy Training!