I found the Ubuntu kernel commit causing this for me. Details are at the end of this reply.
Information for the work-around
Without building kernel with the commit reversed, the work-around with all Focal Fossa 5.4 and 5.6 kernels I tested was using
nvidia_drm.modeset=1. It can be used eg. in the two following ways:
1: Modifying Grub's configuration, so that the parameter is added to kernel command line when booting, by editing the file:
GRUB_CMDLINE_LINUX_DEFAULT="quiet splash nvidia_drm.modeset=1"
After this we need to run
sudo update-grub, to add the parameter to the GRUB configuration at
2: Instead of the kernel command line parameter, the parameter can be defined in modprobe's configuration:
Creating file (name doesn't matter as long as it is in
/etc/modprobe.d and ends
options nvidia_drm modeset=1
nvidia_drm is not built into the kernel, this should be the preferred way.
After reversing this commit, I was able to build latest Focal Fossa 5.4 kernel that did not cause X to freeze. I have also made an bug report about the issue.
This same code is in Bionic kernels also but does not cause X to freeze with them. Something else in the Focal Fossa kernel has changed since Bionic that causes including this commit to freeze X.
Committed by Andrea Righi email@example.com
25.11.2019 14.56.25 +0100
Author: Andy Whitcroft firstname.lastname@example.org
Date: Wed Apr 16 19:40:57 2014 +0100
UBUNTU: SAUCE: vt -- maintain bootloader screen mode and content until vt switch
Introduce a new VT mode KD_TRANSPARENT which endevours to leave the current content of the framebuffer untouched. This allows the bootloader to insert a graphical splash and have the kernel maintain it until the OS splash can take over. When we finally switch away (either through programs like plymouth or manually) the content is lost and the VT reverts to text mode.
Signed-off-by: Andy Whitcroft email@example.com