how test the Translation Support for SMMUv3?

[Nicolin Chen]

On Wed, Jan 03, 2024 at 10:09:34PM +0800, Ben wrote:
> At 2024-01-03 03:51:24, "Nicolin Chen" <nicolinc at nvidia.com> wrote:
> >On Sun, Dec 31, 2023 at 10:18:12PM +0800, Ben wrote:
> >
> >> I am trying your patchset on FVP (Fixed Virtual Platforms) but failed.
> >>
> >> Here is the Host side running on FVP (platform is  rdn1egde).
> >>
> >> master:~# echo 0000:05:00.0 > /sys/bus/pci/devices/0000\:05\:00.0/driver/unbind
> >> master:~# echo 0abc aced > /sys/bus/pci/drivers/vfio-pci/new_id
> >>
> >> when i want to run the QEMU to launch a VM, some failed, like below:
> >>
> >> root at master:/# cat qemu-iommufd.sh
> >> ./build/qemu-system-aarch64 -L /usr/local/share/qemu -object iommufd,id=iommufd0 -machine virt,accel=kvm,gic-version=3,iommu=nested-smmuv3,iommufd=iommufd0 -cpu host -m 256m -nographic -kernel /Image -append "noinintrd nokaslr root=/dev/vda rootfstype=ext4 rw" -drive if=none,file=/busybox_arm64.ext4,id=hd0 -device virtio-blk-device,drive=hd0 -device vfio-pci,host=0000:05:00.0,iommufd=iommufd0,id="test0"
> >> root at master:/# sh qemu-iommufd.sh
> >> WARNING: Image format was not specified for '/busybox_arm64.ext4' and probing guessed raw.
> >>          Automatically detecting the format is dangerous for raw images, write operations on block 0 will be restricted.
> >>          Specify the 'raw' format explicitly to remove the restrictions.
> >> qemu-system-aarch64: -device vfio-pci,host=0000:05:00.0,iommufd=iommufd0,id=test0: vfio 0000:05:00.0: vfio /sys/bus/pci/devices/0000:05:00.0/vfio-dev: failed to load "/sys/bus/pci/devices/0000:05:00.0/vfio-dev/vfio0/dev"
> >>
> >> It looks cannot find the /sys/bus/pci/devices/0000:05:00.0/vfio-dev/vfio0/dev for this device.
> >>
> >> root at master:/# ls -l /sys/bus/pci/devices/0000\:05\:00.0/vfio-dev/vfio0/
> >> total 0
> >> lrwxrwxrwx 1 root root    0 Dec 31 13:29 device -> ../../../0000:05:00.0
> >> drwxr-xr-x 2 root root    0 Dec 31 13:29 power
> >> lrwxrwxrwx 1 root root    0 Dec 31 13:29 subsystem -> ../../../../../../../../class/vfio-dev
> >> -rw-r--r-- 1 root root 4096 Dec 31 13:20 uevent
> >>
> >> any suggestion on that?
> >
> >CONFIG_VFIO_DEVICE_CDEV=y
> >
> >Do you have this enabled in kernel config?
> 
> Thanks your suggestion. Right now I can run the QEMU to launch a VM.
> After assigned a device to VM and  binded  the vfio-pci driver for this device in VM,
> it failed to open "/dev/vfio/x" device file. Any suggestion?
> 
> Here is the log and steps:
> 
> On host side:
> root at master:~# lspci -k
> 02:00.0 Unassigned class [ff00]: ARM Device ff80
>         Subsystem: ARM Device 0000
> 
> echo 13b5 ff80 > /sys/bus/pci/drivers/vfio-pci/new_id
> 
> ./qemu-system-aarch64-iommufd -L /usr/local/share/qemu -object iommufd,id=iommufd0 -machine virt,accel=kvm,gic-version=3,iommu=nested-smmuv3,iommufd=iommufd0 -cpu host -m 256m -nographic -kernel ./Image -append "noinintrd nokaslr root=/dev/vda rootfstype=ext4 rw" -drive if=none,file=./busybox_arm64.ext4,id=hd0 -device virtio-blk-device,drive=hd0 -device vfio-pci,host=0000:02:00.0,iommufd=iommufd0
> 
> 
> On the VM side:
> 
> / # echo 13b5 ff80 > /sys/bus/pci/drivers/vfio-pci/new_id
> / # ./vfio_test 0000:00:02.0
> Failed to open /dev/vfio/1, -1 (No such file or directory)

VM side? You mean in the guest? No, you shouldn't configure
it to a VFIO device in the guest. Just treat it as a native
PCI device and let its driver in the guest kernel probe it.

The "Unassigned class" returned by the lspci running in the
host is likely telling you that your kernel doesn't support
the device at all?

Try some simpler device that's supported first. What happens
to the 0000:05:00.0 that you passed through previously?

Nicolin