Mosh client unable to communicate to mosh server in a guest virtual machine

Question

I'm having trouble trying to connect to mosh-server that is installed in a Arch Linux guest Virtual Machine (set up with the stack: KVM, libvirt and QEMU).

I have a port forward configured with: virsh qemu-monitor-command --hmp arch_test 'hostfwd_add ::2222-:22', therefore I can connect with ssh -p 2222 arch@localhost, without issues.

Issuing the command: mosh --server="LANG=de_DE.UTF-8 mosh-server" --ssh="ssh -p 2222" -p 60001:60001 arch@localhost on the client (host), opens a connection but I'm left with a message saying this:

mosh: Nothing received from server on UDP port 60001. [To quit: Ctrl-^ .]

Naturally, I went to the F.A.Q. to try understand the problem:

Q: What does the message "Nothing received from the server on UDP port 60003" mean?

This means that mosh was able to start mosh-server successfully on the remote machine, but the client is not able to communicate with the server. This generally means that some type of firewall is blocking the UDP packets between the client and the server. If you had to forward TCP port 22 on a NAT for SSH, then you will have to forward UDP ports as well. Mosh will use the first available UDP port, starting at 60001 and stopping at 60999. If you are only going to have a small handful of concurrent sessions on a server, then you can forward a smaller range of ports (e.g., 60000 to 60010).

Tools like netstat, netcat, socat, and tcpdump can be useful for debugging networking and firewall problems.

This problem can also be the result of a bug in glibc 2.22 that affects programs that link with protobuf and utempter and use aggressive compiler hardening flags. (glibc bugtracker entry, as well as Mosh bugtracker entry.) The problem causes mosh-server to segfault immediately on startup. We believe we have worked around this problem in Mosh 1.2.6, but please report a bug if you find otherwise.

After that, I tried the following:

• make sure that I have a UDP port forward, for example: virsh qemu-monitor-command --hmp arch_test 'hostfwd_add udp::60001-:60001';
• make sure that the firewall (if any) has this port open: firewall-cmd --zone=public --add-port=60000-60010/udp (after checking which zone is active);
• try listening with nc in the server (guest) with nc -l -u 60001 and then try connecting to it, in the client (host), with nc -u localhost 60001, and type anything expecting "echoes" in the server, which didn't, but the connection was not broken after hitting enter to send;
• try scanning for open UDP ports with nc -v -z -u -w 5 localhost 60000-60010, which yielded nothing, only for TCP for the 2222 port;

I'm not sure what else to try. Maybe I'm missing some configuration or command, I would appreciate any guidance.

Answer 1

For some reason, that is at this moment unknown to me, the command virsh qemu-monitor-command --hmp archlinux 'hostfwd_add udp::60001-:60001' worked as expected.

I'll retrace me steps:

1. using an Arch Linux cloud image, import qcow2 image, with

❯ virt-install  \
        --name archlinux \
        --memory 4096             \
        --vcpus=2,maxvcpus=4      \
        --cpu host                \
        --disk $HOME/VMs/$(ls | grep 'Arch-Linux'),bus=virtio,format=qcow2  \
        --network user            \
        --virt-type kvm \
      --os-variant archlinux \
      --import

2. then, forward the port to from 2222 (host) to 22 (guest, SSH default port), with virsh qemu-monitor-command --hmp archlinux 'hostfwd_add ::2222-:22';
3. further, log-in into the session and install mosh, with sudo pacman -S mosh;
4. then, install mosh in the same way, on the host;
5. make sure both guest and host have the same locale, you can check out their F.A.Q. about this, for instance, in my case, the host was in German, so I needed to generate the German locale in the guest;
6. port forward the UPD port that mosh expects to connect the client to, which, at this moment of writing is 60001, you can achieve this by issuing the command ❯ virsh qemu-monitor-command --hmp archlinux 'hostfwd_add udp::60001-:60001', where "archlinux" is the name of your guest;
7. finally, connect with the guest, by issuing mosh --ssh="ssh -p 2222" arch@localhost, if you have trouble with the locale, like me, you can add the locale explicitly like so: mosh --server="LANG=de_DE.UTF-8 mosh-server" --ssh="ssh -p 2222" arch@localhost.

And that should be it.

Answer 2

I had a similar problem with Mosh. I solved it by excepting Mosh from my VPN connection. In my case, I added a rule to my VPN client to route my server's IP directly.