Technical Article
Recently, my organization had the need to provide web proxy service to internal users, while not clobbering hotel, home, remote office, coffee shoppe, etc. type access while users were roaming outside of our divisions’ walls. The purpose is to apply content filtering rules to outbound Web traffic based on our organization’s security policy (i.e. no external webmail, personal web storage sites, etc.). I did some research and testing on this side and have come up with a solution that seems to work well across the board for our clients.
Using Internet Explorer’s capability to Automatically detect proxy server settings, IE uses the proxy when the proxy server is reachable, and connects directly when it is not. I have tested this with success (after a lot of initial troubles and debugging ;-)
The components involved in the proposed and tested solution:
- Proxy Auto-configuration file (PAC)
- Web Proxy Automatic Discovery (WPAD)
- Related DHCP and DNS settings
- Group Policies in Active Directory (GPO)
PAC file:
The first step is to configure the Proxy Auto-configuration file (or PAC for short). This is a JavaScript-like file that has a set of predetermined variables and functions for use in making decisions defining the browser’s behavior at runtime. See below for references.
This file can be hard coded in the browser, or preferably delivered using WPAD (see next).
I have built the following sample file using the PAC spec standards which tests for exception sites first (should be accessed directly by the browser) – things like internal sites, private addresses, etc.
Then, the file will test against the client’s IP address (to determine network location). If the IP address is within our internal subnet ranges , it sets the proxy server(s) to use.
The file ends with an else statement that catches all other conditions and sets the browser to use direct access (for when the computer is located outside corporate-controlled facilities).
I have successfully tested this file format with both IE and Firefox. It is provided below as an example for you to utilize, but I make no warranties or claims of fitness-to-purpose. There are many additional testing conditions that might be more relevant to another environment or set of business policies (e.g. Time-of-day, day-of-week, DNS information, etc.).
Sample file:
function FindProxyForURL(url, host)
{
if (
isInNet(host, "10.0.0.0", "255.0.0.0") ||
isInNet(host, "172.16.0.0", "255.240.0.0") ||
isInNet(host, "192.168.0.0", "255.255.0.0") ||
isPlainHostName(host) ||
localHostOrDomainIs(host, "127.0.0.1") ||
dnsDomainIs(host, ".company.local")
)
return "DIRECT";
else if (
isInNet(myIpAddress(), "10.1.0.0", "255.255.0.0") ||
isInNet(myIpAddress(), "10.2.0.0", "255.255.0.0") ||
isInNet(myIpAddress(), "10.3.0.0", "255.255.0.0") ||
isInNet(myIpAddress(), "10.4.0.0", "255.255.0.0") ||
isInNet(myIpAddress(), "10.7.0.0", "255.255.0.0") ||
isInNet(myIpAddress(), "10.9.0.0", "255.255.0.0") ||
isInNet(myIpAddress(), "10.10.0.0", "255.255.0.0") ||
isInNet(myIpAddress(), "169.254.0.0", "255.255.0.0") ||
isInNet(myIpAddress(), "172.16.199.0", "255.255.255.0")
)
return "PROXY prx0.us.company.local:8080;" +
"PROXY prx1.us.company.local:8080";
else return "DIRECT";
}
Note: Thanks to Jay Kulsh for pointing out
RFC1918 compliance in above network tests.
WPAD
Second came the challenge of getting the clients to use that file without hard-setting it (hardcoding almost always an undesirable option if it can be avoided). The mechanism used for this is WPAD, which allows the browser to "discover" where the above configuration file is stored, allowing it to then dynamically pull it down and apply the function code therein during operation. There are several mechanisms available to WPAD, but they center on DHCP and DNS. I have opted to implement both the required elements of the standard (DHCP option and DNS "well known alias" methods), and have left alone the optional requirements as they are redundant for my purposes and remain unused if the required elements exist – if the required elements are unavailable (in my environment), it would be equivalent to a network outage at which point we have bigger problems to solve than WPAD not functioning.
The first step to configure WPAD is to put the PAC file onto a web server for all users to access. Depending on the web server platform and version chosen to host the file, this might require defining additional MIME types to allow the server to properly serve the file (see standard). As a reference point, Win2K’s IIS server generally hosts whatever files you make available, whereas Win2K3 (2003 Server) IIS requires the additional MIME definitions – otherwise you will receive 403 errors, and the browser will transparently fail to pickup the PAC file without displaying an error message (by design).
The recommendation is to place the PAC file on the same server that hosts the proxy. The rationale is that if it is unavailable, implicitly so is the proxy and as such, should not be utilized. However, one might opt to locate the PAC file on a neutral / different server (independent of the proxy) to allow for more robust proxy fail-over (since the PAC standard allows for multiple proxies to be defined for fail-over).
DHCP: The second step is to configure a custom vendor option on the DHCP server. The reserved vendor option for WPAD is 252, and must be created on the DHCP Server config first. Then you can configure scopes (either via a server-wide setting or per-scope setting, relevant to your environment, with the proper URL string which tells the browser where to get the PAC file. However, the DHCP piece is not fully functional for PAC file location until the AutoDetect option is enabled in the web browser. In Firefox, this is the "auto-detect proxy" setting under Tools-Options-Connections – In IE, one can deploy the setting via GPO (see below). The value of the 252 WPAD option is the full URL to the PAC file, including FQDN of the web server (e.g. http://websrv.us.company.local/wpad.dat). This is the first component tried by WPAD for PAC file location, and is a required component of the standard.
DNS: The third configuration change I made was to place a DNS entry (can be an A or CNAME record) which includes a "well-known alias" for the service discovery – in my case – "wpad" without the quotes, which points to the proxy server. I opted for a CNAME record to alias the proxy itself since that is where my PAC file was located and maintaining multiple autonomous A records for the same host is problematic in this case.
The DNS option appears to be the one favored by Firefox, although secondary by IE based on my test results, so I implemented both to cover both browsers more effectively. (Note: In my corporate environment, the Proxy server in use uses proprietary auth mechanisms that Opera does not support, thereby preventing Opera from functioning with my organization’s proxy. This is why no mention of the Opera browser in this Windows-centric platform discussion).
GPO
The setting to have IE use Auto Detection for its Proxy settings is configured in the same place in Group Policy as if one was hard-coding the proxy (Internet Explorer Maintenance) – it is just a different option. This affects only IE at this time, since Firefox is not natively GPO-aware (author note: Efforts are underway to allow Firefox GPO administration, but not covered here – will cover in an upcoming entry) .
In the Internet Explorer Maintenance area:
User Configuration – Windows Settings – Internet Explorer Maintenance
"Automatically detect configuration settings"- Enabled
"Automatic Browser Configuration" – Not configured
This sets IE to use
WPAD to discover the
PAC file.
Additionally, in the Computer Configuration area, the following settings should be changed to maintain consistency and compliance of the browser’s settings:
Computer Configuration – Administrative Templates – System/Group Policy
"Internet Explorer Maintenance policy processing" – Enabled
"Allow processing across a slow network connection" – Enabled
"Do not apply during periodic background processing"- Disabled
"Process even if the Group Policy objects have not changed"- Enabled
These computer-level
GPO settings set and renew the settings continually, even across slow connections, so that if a user has tampered with the IE proxy settings, they will be changed back (although I know this has worked with other configs, I have not had the time to fully test – i.e. tried to break – this setting yet, so I’m not certain how effective it is in overriding, for example, a proxy the user has defined themselves. Obviously, the best practice would be to hide the Connections tab (and perhaps other areas) from the UI to prevent tampering. Additionally, users should not have Admin rights to their machine if an effective policy is to be maintained with any consistency.
Firefox
The methods described in the above sections, when implemented together, support both IE and Firefox. The main hurdle is centrally deploying the Auto-Detect setting to both browser platforms consistently. Additionally, unrelated, anyone using Firefox should upgrade to 1.5.0.1 if they have not already, for security reasons.
Although many organizations do not officially advocate the use of Firefox (since there are still some management/administration hurdles for corporate deployment), I felt that it was important to find a workable solution that fit for both browsers since use of Firefox has become much more prevalent in the past year. Some organizations might choose to limit the use of Firefox altogether for other reasons (such as application standards, etc.), but I wanted to make the solution as browser-agnostic (cross platform) as possible.
I found that Firefox had some unexpected (by me) behavior in the way that it searched for the PAC file. Specifically, the Auto-Discover mechanism seems to always query the configured web server for the filename wpad.dat (instead of proxy.pac as I originally had the 252 DHCP WPAD option configured). IE obeyed my configuration, but Firefox insisted otherwise. Workaround, and the standard generally used by Windows shops anyway, was to make the name the file wpad.dat, update the 252 option accordingly, and then both browsers could automatically discover the file appropriately.
Also, on an unrelated note, I have had some odd, almost random, occurrences of Firefox interoperability problems with the Computer Associates (CA) SCM (Secure Content Manager) Proxy service. In some cases, the user is prompted for an ID, when – in fact – this authentication should be transparent (based on internal domain ID). The same behavior is not exhibited in IE thusfar.
Further research and "development watching" I am taking from this include the emerging capability for Firefox to be administered via GPO as well as an initiative inside the Firefox open-source development community to support MSI installers for corporate deployment and easy updates (that is, from the Mozilla side). There are third parties that have made great gains in both GPO and MSI for Firefox, but as I stated – that is something I’ll get into in an upcoming post.
Conclusion
From the testing I have done so far, both browsers appear to behave as expected once AutoDetection is setup as above. If the PAC file is not reachable using the "AutoDetect" WPAD mechanism, both browsers automatically default to direct, which gets around the issue of hard-coded proxy settings in hotels, WiFi hotspots, etc. Additionally, once the user connects to the VPN, they receive an internal IP address which matches on the PAC rules, and WPAD finds the PAC file to utilize (via the DNS mechanism of WPAD) and begins using the proxy for connections to the Web – thereby applying our corporate policies.
This system of configurations represents a "best effort" to provide technical compliance with our corporate policies with regard to web content filtering. There still exist some unavoidable loopholes, but those should be addressed through policy education to the user community (i.e. they are not allowed to browse the web in remote locations – home, hotel, airport, coffeeshoppes, etc. – without first connecting to the VPN). Adherence to that policy can be assured with monitoring, logging, and other tools.
References
For more technical information about the PAC and WPAD components of this proposed solution, please reference the following links:
PAC – This file’s format (along with some samples) is described here: http://wp.netscape.com/eng/mozilla/2.0/relnotes/demo/proxy-live.html
WPAD – IETF spec for Web Proxy Automatic Discovery – http://www.wrec.org/Drafts/draft-cooper-webi-wpad-00.txt
I hope this was helpful or informative to some of you out there! Good luck and if you have any questions or comments, please use the comment area or email me directly.
_____________________________________________________________
KevFrey
kevfrey@gmail.com
. . . . . .. . . . . .
Here we are again – another unpatched IE bug has just been announced. This one is similar to the issue last Dec. 2005 (described here 
) in that disabling Active Scripting – Internet Explorer’s JavaScript engine – prevents the flaw from being exploited.
. If you click on that icon, the following dialog box should be displayed:
which will walk you through the update.
