Research: The Web Proxy Authentication Problem
One of the security concerns of deploying private proxies is ensuring that anonymous crawlers and port scanner do not gain access to it. Many proxy providers mandate that their users manually white-list their IPs to get access to the proxy but few users have stable static IPs. Other providers support username/password authentication but many clients such as Google Chrome do not natively support this scheme.
I researched a few open-source solutions for bridging this gap, here were my findings:
Problems
- I am trying to use paid proxy X but it requires either IP-based or HTTP Proxy-Auth based authentication, I want a proxy with no auth requirements that works well with browsers.
Projects Explored
Glider
This was my first choices. It’s lightweight, supports round robin rotation between upstream proxies and supports many protocols.
Doesn’t support Auth proxies yet https://github.com/nadoo/glider/issues/15
Go Proxy
I have this running in production and it works well. Very easy to install and seems to support many protocols. Unfortunately, the documentation isn’t clear on upstream proxies.
Filed an issue here https://github.com/snail007/goproxy/issues/112
MITM Proxy
This is the only proxy I got to work with an HTTP upstream. Unfortunately, it injects it’s own certs and doesn’t support non-intercepting proxying for upstreams.
Squid
Squid is heavy, taking ~180MB of memory on my local machine so it probably will not scale for me. Officially, it supports proxy chaining and others have reported success with it. My attempts haven’t worked, it ignores the cache peer rules and connects directly.
Privoxy
Also claims to support authenticated proxies officially. I haven’t gotten it past DNS resolution for an non-authenticated upstream proxy.
3Proxy
Claims to support proxy chaining with username/password authentication out of the box. The documentation isn’t clear on credential config. Issues here https://github.com/z3APA3A/3proxy/issues/165https://github.com/z3APA3A/3proxy/issues/102 https://github.com/z3APA3A/3proxy/issues/20