I'm getting the error relating to port exhaustion on my server ("Only one usage of each socket address (protocol/network address/port) is normally permitted").
So, I checked with netstat -n, and it lists about 84000 connections.
Based on the local port, and the fact that the foreign IP address is myself, I can see that for example 3654 of them relate to a program running on the same server that I killed over 24 hours ago (yesterday morning, in fact). They're all in the TIME_WAIT state, on foreign ports between 49165 and 65533. At least some of the same foreign port numbers show up as local ports too, for example:
TCP <my_ip>:14046 <my_ip>:49165 TIME_WAIT
...
TCP <my_ip>:49165 <other_ip>:80 TIME_WAIT
14046 is the port that a program on my server listens to, and that the program I killed yesterday used to repeatedly connect to. host <other_ip>
gives me something at compute-1.amazonaws.com
, I don't recognise the IP. I can't get anything out of it from my browser. So I can't quite figure out what on that machine would have connected to port 80 on , but there are various things that have run on the same machine recently that consume web services, so I guess it could be a redirect or similar from one of those.
Separate runs of netstat more than 4 minutes apart show that both connections are sticking around in the list.
Is this expected? I thought sockets were supposed to finally close after 4 minutes or so, but does netstat normally list them for much longer? Are these ports really occupied, and if so is there anything I can do to release them? Preferably without rebooting, but if I have to then I have to.
OS is Windows Server Enterprise, running on a Rackspace VM.
[Edit: hmm, should I have asked this on serverfault?]
[Another edit: a reboot seemed to fix it. All the old connections gone, and new connections of the same type disappearing in a few minutes as expected. I would guess therefore that the network stack somehow got into a malfunctioning state.]