Why is VNC (as a protocol, not my particular connection) so slow, especially in 2021? especially compared to things like nvidia gamestream?

Question

This has been a lurking question in the back of my mind for a while -- and one that I just don't understand fully? Unless VNC has had 0 development since it was created, and hasn't kept up with different encoding/decoding/transfer techniques developed over the years.

My scenario is this:

• 2 linux machines are wired to the router
• Linux Machine A (LMA) connects to Linux Machine B (LMB) over VNC
• The VNC session observed from LMB on LMA seems to have about 4-6 frames per second, regardless of quality or color depth and quality.

This is frustrating, because I use nvidia Gamestream for both streaming games at 60fps to my TV in the living room and wirelessly to my VR Headset at 75fps. I also preferred using nvidia gamestream for remote desktop when I had a laptop. Laptop -> Nvidia gamestream to Windows PC -> Linux VM -> remote programming time (it may seem convoluted, but it is waaay more performant and responsive than VNC/RDP)

So, the root of my question:

• why is VNC so bad?
• why is RDP also bad (but slightly better in terms of refresh rate, on Linux it doesn't work the same as VNC, so it's a non-option for me)
• Are the other remote control protocols to use?
• When I've remoted in to my windows Machine I use Moonlight, which uses nvidia gamestream, and it's by far the best remote desktop experience I've had -- I just doubt nvidia will ever support linux hosts for gamestream

---

Proof of "good enough bandwidth" between the two machines, using iperf3

using ssh port-forwarding (same as I use for VNC)

# on the client
❯ ssh -XY -l me -L 5201:localhost:5201 192.168.1.17
# welcome message on the server, etc
❯ iperf3 -s -B 0.0.0.0 -V
Linux Calypso 5.11.0-17-generic #18-Ubuntu SMP Thu May 6 20:10:11 UTC 2021 x86_64
-----------------------------------------------------------
Server listening on 5201

On the client:

❯ iperf3 -c localhost -R -p 5201 -V -t 5
iperf 3.7
Linux Athena 5.4.0-73-generic #82-Ubuntu SMP Wed Apr 14 17:39:42 UTC 2021 x86_64
Control connection MSS 32768
Time: Mon, 31 May 2021 16:10:25 GMT
Connecting to host localhost, port 5201
Reverse mode, remote host localhost is sending
      Cookie: wzbvh5iistp5fq3bzz7iaenrr46ptro3md7y
      TCP MSS: 32768 (default)
[  5] local 127.0.0.1 port 44766 connected to 127.0.0.1 port 5201
Starting Test: protocol: TCP, 1 streams, 131072 byte blocks, omitting 0 seconds, 5 second test, tos 0
[ ID] Interval           Transfer     Bitrate
[  5]   0.00-1.00   sec   108 MBytes   904 Mbits/sec                  
[  5]   1.00-2.00   sec   107 MBytes   897 Mbits/sec                  
[  5]   2.00-3.00   sec   109 MBytes   911 Mbits/sec                  
[  5]   3.00-4.00   sec   112 MBytes   938 Mbits/sec                  
[  5]   4.00-5.00   sec   111 MBytes   933 Mbits/sec                  
- - - - - - - - - - - - - - - - - - - - - - - - -
Test Complete. Summary Results:
[ ID] Interval           Transfer     Bitrate         Retr
[  5]   0.00-5.00   sec   558 MBytes   935 Mbits/sec    0             sender
[  5]   0.00-5.00   sec   546 MBytes   916 Mbits/sec                  receiver
snd_tcp_congestion cubic
rcv_tcp_congestion cubic

iperf Done.

Answer 1

Two reasons:

1. Because VNC is from the days when hardware video compression was not a thing
2. Because "fast" video compression and transmission relies on lossy compression techniques.

VNC is expected to faithfully recreate a remote desktop pixel for pixel over a network connection. It might use CPU based lossless compression techniques to reduce the amount of bandwidth, but in theory it is pixel perfect. Transmitting 1920 x 1080 x 32bits at 60 frames per second in a lossless format is just short of 500MB/s (megabytes, not megabits) which is unrealistic even for "great" modern internet connections. VNC uses techniques such as only sending data blocks that change as a result, on a mostly static desktop with large blocks of similar colour you can get away with full screen refreshes every few seconds and only the changes being transmitted.

Gamestream on the other hand is well aware that you are essentially playing a video (game) that has a lot of visual noise similar to a film, but that you do need to see pretty much every full frame as close as possible to the original. As a result they believe that some level of lossy encoding is acceptable in a tradeoff for cutting down on bandwidth. The kind of codecs used are ones that graphics cards already know how to deal with in hardware using decoders. Creating a hardware encoder and putting it beside the decoder is not that hard a thing.

Gamestream works by leveraging hardware encoders and decoders, but as a result is relatively constant bandwidth and a complete data stream of what was done over time. It can achieve a consistent (low) latency due to everything being done in hardware with most stages being "fire and forget".

The kind of encoding done by Gamestream would, when applied to VNC, result in (most likely) higher bandwidth required due to the type of encoder, and worse quality due to the lossy compression. The tradeoff is that technically VNC and similar solutions such as RDP are "slower" though in practice the delay is quite acceptable over a reasonable link.

Game streaming and remote desktop have completely different intentions and philosophies, though from experience I have used game streaming for a remote desktop in the past and it does work perfectly well, with the caveat of some loss of visual fidelity.

Answer 2

Calling all VNC slow is an over-simplification that doesn't take into account the improvements achieved by some companies such as TightVNC and RealVNC.

While the traditional VNC was slow, although still better than Linux X, it is not as slow nowadays as it was before.

Various flavors of VNC have been taking on features and optimizations from faster implementations such as Microsoft's RDP and picking up speed. They have also improved their usage of network connections.

See for example the article from January 2019 of
RealVNC releases new high-speed streaming for VNC Connect.