2

I set up a webserver on a virtual machine on my home desktop. I have my router forward several ports (80, 443, etc) to my desktop, when then forwards them via NAT to the virtual machine (see diagram for illustration). This seems to work just fine when running a simple apache server. I encounter problems, however, when trying to use WebSockets.

I set up a simple WebSocket echo server (copied exactly from https://www.boost.org/doc/libs/master/libs/beast/example/websocket/server/async-ssl/websocket_server_async_ssl.cpp except for of course changing the SSL certificates to be my own) to listen on port 6164 which I then also forward in the same manner as the others. I also set up a simple web page (code below for reference) hosted on the apache server I mentioned before to open a websocket to the server and send a message.

The server seems to work exactly as intended, I can open a WebSocket connection, and send a message which gets echoed back, etc. However, I have noticed that every time I open the WebSocket from a computer (or other deivce) on my local network, it always uses port 6164 on both sides, and therefore only one connection from my local network can occur at once. This is not the case for connections originating from devices outside of my local network, they use ephemeral ports on the client side. (See port breakdown diagram for tested cases).

  1. Why is this happening?
  2. Are there tools that I could use to track down what is happening to the connection along its full path? (I.E. what IPs/ports is it being forwarded through)
  3. As I am rather new to networking: what determines the client's port anyway? Is it just the browser itself picking one?

A little additional information: when I check netstat, the listener displays

tcp 0 0 0.0.0.0:6164 0.0.0.0:* LISTEN 26866/./serv...

and when any compuer (or my phone) on my network is connected it displays

tcp 0 0 10.0.2.15:6164 10.0.2.2:6164 ESTABLISHED 26866/./serv...

but when I disconnect my phone from my network and use data, or use a computer not on my local network it reads

tcp 0 0 10.0.2.15:6164 10.0.2.2:51616 ESTABLISHED 26866/./serv...

where the 51616 ephemeral port number changes every time. 10.0.2.15 is my VM's IP, 10.0.2.2 is the VM's gateway.

Additionally, this is not unique to port 6164, I tried having the server listen on several other ports, changing the ws = new WebSocket("wss://shaun.ralsum.com:6164"); line in the code accordingly, with the same result.

Edit: Here is an image of my port forwarding rules for router->Desktop (left) and Desktop->VM (right) as requested by user1686.


web page code

<!DOCTYPE HTML>

<html>
  <head>
    
    <script>
      function openConnection(){
      if(window.WebSocket){
          // Open a web socket
          ws = new WebSocket("wss://shaun.ralsum.com:6164");
          ws.onopen = function(){
          ws.send("socket open");
          document.getElementById('socketStatus').innerHTML =
              "socket connected";
          };
          ws.onmessage = function(evt){
          var recievedMessage = evt.data;
          document.getElementById('serverResponse').innerHTML =
              recievedMessage;
          };
      } else{
          document.getElementById('socketStatus').innerHTML =
          "WebSockets not supported";
      }
      }

      function sendMessage(){
      // use global web socket object to send a message
      ws.send("sending another message");
      }
      
      function closeConnection(){
      // close the web socket connection
      ws.close();
      document.getElementById('socketStatus').innerHTML =
          "socket disconnected";
      }
    </script>
    
  </head>
  
  <body>
    <p id="socketStatus">socket disconnected</p>
    <button onclick="openConnection()">create socket</button>
    <button onclick="sendMessage()">send message</button>
    <button onclick="closeConnection()">close socket</button>
    <p id="serverResponse">response from server</p>
  </body>
  
</html>

Note: originally posted to stack overflow but was advised to post here instead.

6
  • 1
    What do all of your port forwarding rules look like? Commented Apr 21, 2021 at 1:34
  • @user1686 I added a screenshot of my port forwarding rules to the post.
    – SAlsum
    Commented Apr 21, 2021 at 2:15
  • 1
    Check netstat on the client side. It’s probably not the client device doing this. It is the port forwarding / NAT mechanism that is doing it. The client probably uses a random port as expected. Commented Apr 21, 2021 at 3:01
  • @Appleoddity Alright, I checked netstat on the client for multiple different cases (checked each multiple times to be sure): (1) Separate computer on my LAN: 10.0.0.15:[ephemeral]->[router ip]:6164 (2) Desktop hosting the VM: 192.168.1.147:6164->dsldevice:6164 (3) VM hosting the server itself: 10.0.2.15:[ephemeral]->[router ip]:6164 So it seems you are right (except somehow in the desktop case?) but what could be causing the ephemeral to be overridden in the LAN case but not the external case?
    – SAlsum
    Commented Apr 21, 2021 at 3:42
  • Quite possibly it is a problem with the router. You are doing hairpin NAT it looks like. This means you are connecting to the router’s public IP from inside your local network, and the router is NATing the connection back in to the local network. This is problematic in multiple cases. It would not surprise me if the router simply has bugs. Connect directly to the server’s private IP and avoid using the public IP inside the network. This can be accomplished with split-dns as well if the router will allow you to add static DNS entries. Commented Apr 21, 2021 at 3:57

0

You must log in to answer this question.

Browse other questions tagged .