0

I have a Dell Latitude E6530 laptop with Optimus graphics (Intel/Nvidia) and I am attempting a dGPU passthrough that I have read countless articles and forum threads about trying to overcome this roadblock. After much struggling with configs and mismatched packages, I finally have the vfio_pci module loading on boot and the vfio-pci driver bound to my dGPU and it's Audio Device. However, I cannot seem to get rid of the infamous Error 43 that prevents my dGPU from actually loading in the VM. I have tried the XML vendor_id workaround, virsh-patcher's error-43 workaround, disabling hyperv features in the XML, and various other XML edits that have been reported to fix this issue for others. In the end, I haven't gotten anywhere. I also dumped my dGPU's vBIOS using the VBiosFinder tool on Github and added it into my XML, however it no longer booted after that even with QXL as the main video driver. I will list outputs here for info and any more can be provided if needed.

lspci -nnk -s 1:

01:00.0 VGA compatible controller [0300]: NVIDIA Corporation GF108GLM [NVS 5200M] [10de:0dfc] (rev a1)
    Subsystem: Dell GF108GLM [NVS 5200M] [1028:1535]
    Kernel driver in use: vfio-pci
    Kernel modules: nouveau
01:00.1 Audio device [0403]: NVIDIA Corporation GF108 High Definition Audio Controller [10de:0bea] (rev a1)
    Subsystem: Dell GF108 High Definition Audio Controller [1028:1535]
    Kernel driver in use: vfio-pci
    Kernel modules: snd_hda_intel

lsmod|grep vfio

vfio_pci               57344  0
irqbypass              16384  2 vfio_pci,kvm
vfio_virqfd            16384  1 vfio_pci
vfio_iommu_type1       36864  0
vfio                   36864  2 vfio_iommu_type1,vfio_pci

/etc/default/grub

GRUB_CMDLINE_LINUX_DEFAULT="quiet intel_iommu=on,igfx_off iommu=pt pcie_acs_override=downstream vfio-pci.ids=10de:0dfc,10de:0bea apparmor=1 security=apparmor udev.log_priority=3"

/etc/mkinitcpio.conf

MODULES="vfio vfio_iommu_type1 vfio_pci vfio_virqfd vhost-net"
FILES=""
HOOKS="base udev autodetect modconf block keyboard keymap filesystems"

journalctl -b|grep -i vfio

Apr 19 18:54:41 lions-den kernel: Command line: BOOT_IMAGE=/boot/vmlinuz-5.4-x86_64 root=UUID=9aadff70-70e9-4adf-ac3c-e9b737d68ee4 rw quiet intel_iommu=on,igfx_off iommu=pt pcie_acs_override=downstream vfio-pci.ids=10de:0dfc,10de:0bea apparmor=1 security=apparmor udev.log_priority=3
Apr 19 18:54:41 lions-den kernel: Kernel command line: BOOT_IMAGE=/boot/vmlinuz-5.4-x86_64 root=UUID=9aadff70-70e9-4adf-ac3c-e9b737d68ee4 rw quiet intel_iommu=on,igfx_off iommu=pt pcie_acs_override=downstream vfio-pci.ids=10de:0dfc,10de:0bea apparmor=1 security=apparmor udev.log_priority=3
Apr 19 18:54:41 lions-den kernel: VFIO - User Level meta-driver version: 0.3
Apr 19 18:54:41 lions-den kernel: vfio-pci 0000:01:00.0: vgaarb: changed VGA decodes: olddecodes=io+mem,decodes=io+mem:owns=none
Apr 19 18:54:41 lions-den kernel: vfio_pci: add [10de:0dfc[ffffffff:ffffffff]] class 0x000000/00000000
Apr 19 18:54:41 lions-den kernel: vfio_pci: add [10de:0bea[ffffffff:ffffffff]] class 0x000000/00000000
Apr 19 18:54:45 lions-den kernel: vfio-pci 0000:01:00.0: vgaarb: changed VGA decodes: olddecodes=io+mem,decodes=io+mem:owns=none
Apr 19 18:54:45 lions-den kernel: vfio-pci 0000:01:00.0: optimus capabilities: enabled, status dynamic power, hda bios codec supported

sudo virsh dumpxml win10



<domain type='kvm' xmlns:qemu='http://libvirt.org/schemas/domain/qemu/1.0'>
  <name>win10</name>
  <uuid>6d92b905-bb5e-48cf-92da-ae1584aa12a3</uuid>
  <metadata>
    <libosinfo:libosinfo xmlns:libosinfo="http://libosinfo.org/xmlns/libvirt/domain/1.0">
      <libosinfo:os id="http://microsoft.com/win/10"/>
    </libosinfo:libosinfo>
  </metadata>
  <memory unit='KiB'>10485760</memory>
  <currentMemory unit='KiB'>10485760</currentMemory>
  <vcpu placement='static' current='7'>8</vcpu>
  <cputune>
    <vcpupin vcpu='0' cpuset='0'/>
    <vcpupin vcpu='1' cpuset='4'/>
    <vcpupin vcpu='2' cpuset='1'/>
    <vcpupin vcpu='3' cpuset='5'/>
    <vcpupin vcpu='4' cpuset='2'/>
    <vcpupin vcpu='5' cpuset='6'/>
    <vcpupin vcpu='6' cpuset='3'/>
  </cputune>
  <os>
    <type arch='x86_64' machine='pc-q35-4.2'>hvm</type>
    <loader readonly='yes' type='pflash'>/usr/share/ovmf/x64/OVMF_CODE.fd</loader>
    <nvram>/var/lib/libvirt/qemu/nvram/win10_VARS.fd</nvram>
    <boot dev='hd'/>
  </os>
  <features>
    <acpi/>
    <apic/>
    <hyperv>
      <relaxed state='on'/>
      <vapic state='on'/>
      <spinlocks state='on' retries='8191'/>
      <vendor_id state='on' value='1234567890ab'/>
    </hyperv>
    <kvm>
      <hidden state='on'/>
    </kvm>
    <vmport state='off'/>
    <ioapic driver='kvm'/>
  </features>
  <cpu mode='host-passthrough' check='none'>
    <topology sockets='1' cores='4' threads='2'/>
  </cpu>
  <clock offset='localtime'>
    <timer name='rtc' tickpolicy='catchup'/>
    <timer name='pit' tickpolicy='delay'/>
    <timer name='hpet' present='no'/>
    <timer name='hypervclock' present='yes'/>
  </clock>
  <on_poweroff>destroy</on_poweroff>
  <on_reboot>restart</on_reboot>
  <on_crash>destroy</on_crash>
  <pm>
    <suspend-to-mem enabled='no'/>
    <suspend-to-disk enabled='no'/>
  </pm>
  <devices>
    <emulator>/usr/bin/qemu-system-x86_64</emulator>
    <disk type='file' device='disk'>
      <driver name='qemu' type='raw'/>
      <source file='/home/rebel/libvirt/images/win10.img'/>
      <target dev='vda' bus='virtio'/>
      <address type='pci' domain='0x0000' bus='0x08' slot='0x00' function='0x0'/>
    </disk>
    <disk type='file' device='cdrom'>
      <driver name='qemu' type='raw'/>
      <source file='/home/rebel/ISOs/Win10_1909_English_x64.iso'/>
      <target dev='sdb' bus='sata'/>
      <readonly/>
      <address type='drive' controller='0' bus='0' target='0' unit='1'/>
    </disk>
    <disk type='file' device='cdrom'>
      <driver name='qemu' type='raw'/>
      <source file='/home/rebel/ISOs/virtio-win-0.1.173.iso'/>
      <target dev='sdc' bus='sata'/>
      <readonly/>
      <address type='drive' controller='0' bus='0' target='0' unit='2'/>
    </disk>
    <controller type='usb' index='0' model='qemu-xhci' ports='15'>
      <address type='pci' domain='0x0000' bus='0x03' slot='0x00' function='0x0'/>
    </controller>
    <controller type='sata' index='0'>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x1f' function='0x2'/>
    </controller>
    <controller type='pci' index='0' model='pcie-root'/>
    <controller type='pci' index='1' model='pcie-root-port'>
      <model name='pcie-root-port'/>
      <target chassis='1' port='0x10'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x0' multifunction='on'/>
    </controller>
    <controller type='pci' index='2' model='pcie-root-port'>
      <model name='pcie-root-port'/>
      <target chassis='2' port='0x11'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x1'/>
    </controller>
    <controller type='pci' index='3' model='pcie-root-port'>
      <model name='pcie-root-port'/>
      <target chassis='3' port='0x12'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x2'/>
    </controller>
    <controller type='pci' index='4' model='pcie-to-pci-bridge'>
      <model name='pcie-pci-bridge'/>
      <address type='pci' domain='0x0000' bus='0x01' slot='0x00' function='0x0'/>
    </controller>
    <controller type='pci' index='5' model='pcie-root-port'>
      <model name='pcie-root-port'/>
      <target chassis='5' port='0x13'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x3'/>
    </controller>
    <controller type='pci' index='6' model='pcie-root-port'>
      <model name='pcie-root-port'/>
      <target chassis='6' port='0x14'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x4'/>
    </controller>
    <controller type='pci' index='7' model='pcie-root-port'>
      <model name='pcie-root-port'/>
      <target chassis='7' port='0x15'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x5'/>
    </controller>
    <controller type='pci' index='8' model='pcie-root-port'>
      <model name='pcie-root-port'/>
      <target chassis='8' port='0x16'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x6'/>
    </controller>
    <controller type='pci' index='9' model='pcie-root-port'>
      <model name='pcie-root-port'/>
      <target chassis='9' port='0x17'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x7'/>
    </controller>
    <controller type='virtio-serial' index='0'>
      <address type='pci' domain='0x0000' bus='0x05' slot='0x00' function='0x0'/>
    </controller>
    <interface type='network'>
      <mac address='52:54:00:ad:bc:10'/>
      <source network='default'/>
      <model type='virtio'/>
      <address type='pci' domain='0x0000' bus='0x02' slot='0x00' function='0x0'/>
    </interface>
    <channel type='spicevmc'>
      <target type='virtio' name='com.redhat.spice.0'/>
      <address type='virtio-serial' controller='0' bus='0' port='1'/>
    </channel>
    <input type='tablet' bus='usb'>
      <address type='usb' bus='0' port='1'/>
    </input>
    <input type='mouse' bus='ps2'/>
    <input type='keyboard' bus='ps2'/>
    <graphics type='spice' autoport='yes'>
      <listen type='address'/>
      <image compression='off'/>
      <gl enable='no' rendernode='/dev/dri/by-path/pci-0000:00:02.0-render'/>
    </graphics>
    <sound model='ich9'>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x1b' function='0x0'/>
    </sound>
    <video>
      <model type='qxl' ram='65536' vram='65536' vgamem='16384' heads='1' primary='yes'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x0'/>
    </video>
    <hostdev mode='subsystem' type='pci' managed='yes'>
      <source>
        <address domain='0x0000' bus='0x01' slot='0x00' function='0x0'/>
      </source>
      <rom bar='on' file='/home/rebel/libvirt/vbios_10de_0dfc.rom'/>
      <address type='pci' domain='0x0000' bus='0x07' slot='0x00' function='0x0'/>
    </hostdev>
    <hostdev mode='subsystem' type='pci' managed='yes'>
      <source>
        <address domain='0x0000' bus='0x01' slot='0x00' function='0x1'/>
      </source>
      <address type='pci' domain='0x0000' bus='0x09' slot='0x00' function='0x0'/>
    </hostdev>
    <watchdog model='i6300esb' action='shutdown'>
      <address type='pci' domain='0x0000' bus='0x04' slot='0x05' function='0x0'/>
    </watchdog>
    <memballoon model='virtio'>
      <address type='pci' domain='0x0000' bus='0x06' slot='0x00' function='0x0'/>
    </memballoon>
  </devices>
  <qemu:commandline>
    <qemu:arg value='-audiodev'/>
    <qemu:arg value='pa,id=snd0,server=unix:/tmp/pulse-socket'/>
  </qemu:commandline>
</domain>

2 Answers 2

0

One of the things you may be missing is a fake battery ACPI table entry. See here for details: https://github.com/jscinoz/optimus-vfio-docs/issues/2

Nvidia driver does various checks to determine whether the customer has paid enough for their GPU (i.e. bought a Quadro) before they are deemed worthy of having that GPU run in a virtual machine. They used to check just for signs of a hypervisor, but for laptop 10xx and 20xx series GPUs they also check for presence of a battery.

0

I faced the same issue, but this fixed it for me.

If you still have code 43 check dmesg for memory reservation errors after starting up VM, if you have similar it could be the case:

$ dmesg | grep BAR
vfio-pci 0000:09:00.0: BAR 3: cannot reserve [mem 0xf0000000-0xf1ffffff 64bit pref]

Find out a PCI Bridge your graphic card is connected to. This will give actual hierarchy of devices:

$ lspci -t

Before starting VM run following lines replacing IDs with actual from previous output.

# echo 1 > /sys/bus/pci/devices/0000\:00\:03.1/remove
# echo 1 > /sys/bus/pci/rescan

For example, my lspci -t returned the following, indicating my GPU was attached to PCI bridge device 0000:00:03.1. Change the above command to match your PCI bridge device, if applicable.

> -[0000:00]-+-00.0
>            +-00.2
>            +-01.0
>            +-01.1-[01]----00.0
>            +-01.3-[03-25]--+-00.0
>            |               +-00.1
>            |               \-00.2-[20-25]--+-00.0-[21]--
>            |                               +-01.0-[22]----00.0
>            |                               \-04.0-[25]--+-00.0
>            |                                            \-00.1
>            +-02.0
>            +-03.0
>            +-03.1-[26]--+-00.0    <--- My GPU at PCI address 26
>            |            +-00.1
>            |            +-00.2
>            |            \-00.3

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .