I have a Gigabyte GA-970A-DS3P board with a AMD FX(tm)-8350 cpu. I have IOMMU enabled in the BIOS.
For the non-XEN linux boot I pass
iommu=pt amd_iommu_dump as a kernel argument and can verify that AMD-V is loaded via
dmsg | grep AMD-V.
When running a QEMU/KVM guest I can passthrough the Marvell 88SE9230 card to the guest and everything works fine.
I want to get this pass-through working with a XEN setup as well. However from the 4 SATA 0-3 ports that the 88SE9230 card supplies only the last port is actually discovering a disk (i.e. if port 0 and 1 are connected, then disk on port 1 is discovered, while port 0 remains empty).
I now run
Xen 4.9 via
apt-get install xen-hypervisor-4.9-amd64 in an Ubuntu-18.04 dom0. Kernel is
4.15.0-29-generic. I rebind my device (in my case 0000:01:00.0) via pciback.sh as described here and rebind via
pciback.sh "0000:01:00.0". Then I can attach 0000:01:00.0 to the Xen guest.
Inside the guest
0000:01:00.0 appears and
pciconf (in my case the XEN guest is FreBSD FreeNAS) works correctly. However only the disk on the last connected port is detected. All other ports seems to be empty even if I have a disk connected. In the KVM version these disks are recognized.
I was wondering where the AMD iommu handling is implemented in the Xen case. The Ubuntu 18.04 dom0 doesnt have any iommu entries in sysfs and the AMD-V debug messages are not printed. So I guess there is a baremetal XEN hypervisor that implements the iommu initialization. Is there some option I need to add to enable AMD iommu handling in the hypervisor (equivalent to the linux
iommu=pt option) and how is this done?