softdep for vfio-pci wont work

Question

I'm trying to do passthrough so my devices are grabbed by vfio-pci before any other module. Here's my /etc/modprobe.d/01-vfio-pci.conf:

softdep nouveau pre: vfio-pci
softdep nvidiafb pre: vfio-pci
softdep snd_hda_intel pre: vfio-pci
softdep nvidia-gpu pre: vfio-pci
softdep i2c_nvidia_gpu pre: vfio-pci
softdep xhci_hcd pre: vfio-pci
options vfio-pci ids=10de:2184,10de:1aeb,10de:1aec,10de:1aed,1022:145f

but here are the devices after boot:

07:00.0 VGA compatible controller [0300]: NVIDIA Corporation TU116 [GeForce GTX 1660] [10de:2184] (rev a1) (prog-if 00 [VGA controller])
    Subsystem: NVIDIA Corporation TU116 [GeForce GTX 1660] [10de:1324]
    Flags: fast devsel, IRQ 10
    Memory at f6000000 (32-bit, non-prefetchable) [disabled] [size=16M]
    Memory at d0000000 (64-bit, prefetchable) [disabled] [size=256M]
    Memory at e0000000 (64-bit, prefetchable) [disabled] [size=32M]
    I/O ports at f000 [disabled] [size=128]
    Expansion ROM at f7000000 [disabled] [size=512K]
    Capabilities: <access denied>
    Kernel driver in use: vfio-pci
    Kernel modules: nvidiafb, nouveau

07:00.1 Audio device [0403]: NVIDIA Corporation Device [10de:1aeb] (rev a1)
    Subsystem: NVIDIA Corporation Device [10de:1324]
    Flags: fast devsel, IRQ 11
    Memory at f7080000 (32-bit, non-prefetchable) [disabled] [size=16K]
    Capabilities: <access denied>
    Kernel driver in use: vfio-pci
    Kernel modules: snd_hda_intel

07:00.2 USB controller [0c03]: NVIDIA Corporation Device [10de:1aec] (rev a1) (prog-if 30 [XHCI])
    Subsystem: NVIDIA Corporation Device [10de:1324]
    Flags: fast devsel, IRQ 47
    Memory at e2000000 (64-bit, prefetchable) [size=256K]
    Memory at e2040000 (64-bit, prefetchable) [size=64K]
    Capabilities: <access denied>
    Kernel driver in use: xhci_hcd

07:00.3 Serial bus controller [0c80]: NVIDIA Corporation Device [10de:1aed] (rev a1)
    Subsystem: NVIDIA Corporation Device [10de:1324]
    Flags: bus master, fast devsel, latency 0, IRQ 61
    Memory at f7084000 (32-bit, non-prefetchable) [size=4K]
    Capabilities: <access denied>
    Kernel driver in use: nvidia-gpu
    Kernel modules: i2c_nvidia_gpu

08:00.3 USB controller [0c03]: Advanced Micro Devices, Inc. [AMD] Zeppelin USB 3.0 Host controller [1022:145f] (prog-if 30 [XHCI])
    Subsystem: Gigabyte Technology Co., Ltd Zeppelin USB 3.0 Host controller [1458:5007]
    Flags: bus master, fast devsel, latency 0, IRQ 48
    Memory at f7200000 (64-bit, non-prefetchable) [size=1M]
    Capabilities: <access denied>
    Kernel driver in use: xhci_hcd

As you can see, only some are grabbed. But they're not grabbed by this modprobe rule I added, it works without it.

lz@z:~$ cat /etc/initramfs-tools/modules
# List of modules that you want to include in your initramfs.
# They will be loaded at boot time in the order below.
#
# Syntax:  module_name [args ...]
#
# You must run update-initramfs(8) to effect this change.
#
# Examples:
#
# raid1
# sd_mod
vfio 
vfio_iommu_type1 
vfio_virqfd 
options vfio_pci ids=10de:2184,10de:1aeb,10de:1aec,10de:1aed,1022:145f
vfio_pci ids=10de:2184,10de:1aeb,10de:1aec,10de:1aed,1022:145f
vfio_pci

lz@z:~$ cat /etc/modules
# /etc/modules: kernel modules to load at boot time.
#
# This file contains the names of kernel modules that should be loaded
# at boot time, one per line. Lines beginning with "#" are ignored.

vfio 
vfio_iommu_type1 
vfio_pci ids=10de:2184,10de:1aeb,10de:1aec,10de:1aed,1022:145f

Answer 1

USB module xhci_hcd is in kernel. It is loaded before modules.

This should work in grub as kernel parameter:

GRUB_CMDLINE_LINUX_DEFAULT="intel_iommu=on modprobe.blacklist=xhci_hcd"

Answer 2

I could not get Ubuntu 20.04 (cloud image) to run vfio-pci for many days. It turned out that none /etc/modules /etc/initramfs-tools/ /etc/modprobe.d/*.conf had any affect. Vfio modules are already built into the kernel. I think the blacklist files might have worked. But for setting the pci ids, only the grub kernel args worked.. and here's the kicker: only worked with vfio-pci.ids not vfio-pci ids. AH! ..back to life finally. :)