How to utilize GPU for/in VNC?

Question

So I got a server that has an Intel processor featuring an iGPU (Intel Xeon E3-1245 v2) which I want to use. I installed everything so that when I SSH into the machine "vainfo" works - so there is at least something working. Inside the VNC session "vainfo" does not work and applications that I start over the VNC session do not utilize the GPU which I can monitor using "sudo intel_gpu_top" via SSH. If I run "sudo vainfo" inside the VNC session, it works.

I tried to playback videos in a browser in VNC and it stutters/lags completely, basically the FPS go way down. The CPU usage goes up and the GPU is not working at all.

Now my question is: Is there a way to get this to work or is this simply impossible? I remember using a VNC connection to another server with a discrete GPU and it worked perfectly via VNC. Barely any CPU usage when playing back videos and that stutter/lag in the VNC connection was barely noticeable. So I'm sure there is a way to do this but I cannot (for days now) figure out what I need to do differently.

I tried different linux distributions, tried different Intel GPU drivers (I figured in the end that I need i965), different VNC applications and different configurations but nothing seems to work. I'm on Debian 11 right now but also tried Ubuntu 18.04, 20.04 and 22.04.

Some stuff I did:

• added the user to the video and render group
• created a /etc/X11/xorg.conf.d/10-intel.conf file containing

Section "OutputClass"
Identifier "Intel"
MatchDriver "i915"
Driver "intel"
EndSection

and many other combinations, including setting same in /etc/X11/xorg.conf

• Added xdg.force_integrated=1 to GRUB_CMDLINE_LINUX_DEFAULT and applied using sudo update-grub

• Tried the following packages

sudo apt install xserver-xorg-video-intel
sudo apt install xserver-xorg-video-all
sudo apt-get install i965-va-driver vainfo
sudo apt-get install libvdpau-va-gl1 i965-va-driver vdpauinfo

• Set VDPAU_DRIVER=va_gl

• Removed nomodeset from GRUB_CMDLINE_LINUX_DEFAULT and applied using sudo update-grub

And I mean as vainfo somehow works, I was not unsuccessful in getting the iGPU to generally work or let's say be ready to work but that's it.

The only way I was able to utilize the GPU for a complete desktop and all running apps was using Windows and connecting via RDP but that's not an option, I just wanted to make sure that the iGPU generally works and is not incompatible with the mainboard or disabled in BIOS.

If anybody could guide me in the right direction, that would be great. Thanks in advance.

Answer 1

Are you running VNC in service mode?

Virtual Mode sessions are rendered by the CPU and can't use hardware acceleration.

If you need hardware acceleration, try using Service Mode instead: sudo systemctl enable vncserver-x11-serviced --now and connect on port 5900.
Note: you will need a connected monitor in most cases for the OS to render the desktop correctly.

In general, if you are looking for performance, VNC is not the place to get it. Have you tried X11 forwarding? Believe it or not, X windows WAS MADE for seamless forwarding. You should try it.

In your question.. it seems like you might not realize that the speeds that VNC can broadcast is limited by the speed of your your network and that a graphics card can take very little advantage of that.

Are you sure this new machine is operating at the network speeds the old was was?

One more addendum to this topic: The performance differences between these remote technologies is not as black and white as I professed above. I do not believe my statements to be wrong, but I don't believe them to be absolute either.