7

We are using a Windows Service to make very frequent HTTP calls to an internal REST service (20-40 calls per second), but notice a long delay in getting responses after the service runs for several minutes.

Looking at netstat, there are quite a few ports with "TIME_WAIT" status, and it seems we may be running out of ports.

How can we ensure that ports are reused?

Improve this question
6
  • Are you disposing request instances after they complete?
    – Wiktor Zychla
    Commented Nov 10, 2015 at 14:42
  • Do you use many threads aswell?
    – CSharpie
    Commented Nov 10, 2015 at 14:44
  • Really this is a performance question... you have zeroed in on this as the cause of your problems... I'm not convinced it is. I'd say that it is far more likely to be a symptom of a different problem.
    – spender
    Commented Nov 10, 2015 at 14:50
  • Sharing the code that you're using to issue the requests is pretty much required to understand whether you're doing something that prevents reuse of connections.
    – EricLaw
    Commented Nov 10, 2015 at 14:56
  • @user2966445 If you're issuing 20-40 calls/sec and they take no more than one second to complete, then I'd encourage you to set the connection limit to about the 50-60 range. If you're still bottlenecking, I'd suggest that your problem lies elsewhere and that raising the connection limit is not the answer to your problem (for instance request limits on the server causing queueing, failure to correctly terminate and dispose of requests so the connections can be reused and so on...)
    – spender
    Commented Nov 10, 2015 at 14:59

3 Answers 3

Reset to default
3

There is a limit in the number of simultaneous outgoing HTTP connections. You can control this by using the System.Net.ServicePointManager.DefaultConnectionLimit static property before creating the HttpWebRequest objects

It might be worthwhile setting this to a higher value than the default which I believe is 2.

If this does not help then you can also increase the default ThreadPool size to allow you to create more requests quicker. The thread pool only ramps up its number of threads gradually - a new thread per every half second, IIRC

Improve this answer
3
  • Thanks, but we already have the following set in the web.config file: <system.net> <connectionManagement> <clear/> <add address="*" maxconnection="1000000" / </connectionManagement> </system.net>
    – user2966445
    Commented Nov 10, 2015 at 14:30
  • Also, we ran a trace while the requests are submitted, and GetRequestStream() averages 0-1 ms, while the GetResponse() method takes sometimes 60,000ms+. We can see on the receiving side the time to process a response is less than 1,000ms, so the bottleneck is on the sending side.
    – user2966445
    Commented Nov 10, 2015 at 14:36
  • @user2966445 Do you ensure you dispose of any reponses etc. when you are finished with them? If you do this then C# by default should reuse the same underlying network connection by default(not the same request however which isn't possible).
    – James Moore
    Commented Nov 10, 2015 at 14:37
1

How can we ensure that ports are reused? Not set the connection limit to a value that almost guarantees that they won't be.

It looks like someone has monkeyed with the ServicePointManager at some point. I'd limit the ServicePoint for this origin: to encourage http pipelining and connection reuse:

ServicePointManager.FindServicePoint(Uri).ConnectionLimit = someSensibleValue;
Improve this answer
2
  • We've already tried setting max connections in the config file: <system.net> <connectionManagement> <clear/> <add address="*" maxconnection="1000000" /> </connectionManagement> </system.net>
    – user2966445
    Commented Nov 10, 2015 at 14:31
  • @user2966445 So, ephemeral ports range from 49152 to 65535... or 16383. It's conceivable that you might exhaust using such a high value. Why do you need to go above 10 or so for this host? Optimally, how many simultaneous unresponded requests do you have? You shouldn't need many more connections than this value under usual conditions. Connections can be reused (unless the remote host is http1.0, issuing connection:close headers, or killing the connection). Such a high value will be detrimental because there will be no reuse of the connections and they'll expire and close to TIME_WAIT.
    – spender
    Commented Nov 10, 2015 at 14:38
1

It's probably related to the same issue that many users run into when using the HttpClient class and classes that use the HttpClient internally. As a result of the HttpClient implementing the IDisposable interface, you'll find many developers having the urge to wrap any new instances of the class in a using statement. There is just one problem with that; When the HttpClient object gets disposed of the attached ports remain blocked for 5 minutes 'TIME_WAIT' until they get released by the OS.

I usually use a single HttpClient instance (singleton) and use fully qualified URLs in combination with asynchronous calls.

Improve this answer

Not the answer you're looking for? Browse other questions tagged or ask your own question.