Understanding Proxy Servers (Operating System, Browsers, JVM, Local Proxy)

Question

In my years of working behind a firewall at my day job, I've lost count on how many times I've had to configure, reconfigure and just plain experiment with proxy server configurations to get that app to work just right.

My guess is that many other developers/technical professionals out there do the same thing. Picture yourself working at a company where you have [gasp] Microsoft ISA server deployed along with Active Directory which seamlessly integrates with it. Of course you're on Windows. Because of that you may have some applications that don't allow to send NTLM credentials to the server or you prefer not to store your own credentials in code somewhere. Thus you have also installed your own proxy that uses the ISA as an upchain proxy - CNTLM.

So here's some examples:

• Internet Explorer - your IT Support folks force the ISA Proxy server and exceptions (which you can't update) into IE's setting via Group Policy. This works flawlessly, except when it doesn't.

• Firefox - Ok, so who still uses Internet Explorer? Firefox - I've used FoxyProxy which works some of the time. Based on the website you're visiting, trial-and-error to pick a proxy (ISA, Direct, CNTLM)

• Other Browsers - If Firefox is not your thing, maybe you use Chrome, Opera, ??.

• Anything Java - How many times have you typed out -Dhttp.proxyHost, -Dhttp.proxyPort?

• IDE's - Anyone else ever seen that either Eclipse update works OR your code that reaches out to the Internet works, but NEVER both?

• Other - VPN Clients, background update processes, SSH, Xmarks (and other browser plugins), etc....

So my question is this: Can you make sense of all of this? What solution have you come up with to get over proxy server hell? Do you daisy-chain multiple proxy's like I have tried to do? I've configured applications with various combinations of CNTLM, Direct, ISA, Windows system level proxy setter: proxycfg, Microsoft Firewall Client, etc... What have you done to overcome this?

Answer 1

I would say the problem is "all windows" and specially ISA... lets take the first example:

Internet Explorer - your IT Support folks force the ISA Proxy server and exceptions (which you can't update) into IE's setting via Group Policy. This works flawlessly, except when it doesn't.

Exactly... *when it doesn't is not so rare"...

I think the main problem is that companies that are too attached to MS are afraid to move away from it or they don't have anyone with the skills to do it...

The best proxy/cache software is IMHO SQUID! Also if possible proxy can be transparent and this way you don't need to set it in IE or other apps..

Now does squid solves all the problems? well not just like that... One problem will be again the NTLM auth... but squid acl's would solve this... For example you could have squid setup to allow non-auth access to windows updates and even cache it (no need for WSUS :) ) same for you're anti virus and java apps...

The config would have to match the scenario of course but squid is indeed very flexible and reliable.

But the most important part would perhaps be the "transparent proxy setup".

What I advice is to setup a linux/unix box has you're firewall/default gateway (you could have OpenVPN server here to handle VPN connections). Squid could be in the same box or in another, a different box is always better.

Of course this requires some knowledge of iptables (or you could use something like shorewall) or packet filter, or you could still have an ISA box with OpenVPN and squid setup in a different box, but why pay for extra licenses, and most important, why run a GUI server for a firewall, vpn or proxy/cache??? But in any case its possible.

NOTE: I'm not trying to say that Windows totally sucks and linux/unix is way better, I do believe that linux/unix is allays better for servers, but its still difficult to find companies willing to have their desktops without windows, mainly because of Office. My point is that in the "servers" world their are other solution a part from Microsoft that might cost much less or even nothing (except the time to set it up ofc)