auto proxy discovery

[rungekutta]

Hi all,

I've been trying to get auto discovery of my web proxy to work on my LAN with limited success and I'm not sure how to debug it so wondering if there are success stories out there or if this is just intrinsically flaky with a diverse set of clients...?

My test client is Mac OS X 10.14.3. I have followed the guide, although not managing to make it work, experimented a bit further. My current config in OPNSense generates this file:

Code: [Select]

/*
  PAC file created via OPNsense
  To use this file you have to enter its URL into your browsers network settings.
*/
function FindProxyForURL(url, host) {

if (!((isPlainHostName(host)) || (shExpMatch(host, "*.mydomain.com")))) {
return "PROXY 192.168.200.1:3128";
}

   // If no rule exists - use a direct connection
   return "DIRECT";
}

... which I think should work. I have added an option to the DHCP server to send the URL on field 252 according to the instructions.

Indeed, on the client:

Code: [Select]

$ scutil --proxy
<dictionary> {
  ExceptionsList : <array> {
    0 : *.local
    1 : 169.254/16
  }
  FTPPassive : 1
  HTTPEnable : 0
  HTTPSEnable : 0
  ProxyAutoConfigEnable : 1
  ProxyAutoConfigURLString : http://192.168.200.1/wpad.dat
  ProxyAutoDiscoveryEnable : 1
}

... looks promising (the URL is picked up from DHCP).

HOWEVER. Safari completely ignores the proxy setting and just loads pages direct instead. Chrome on the other hand actually honours the proxy setting... unless in "incognity mode" in which case it also ignores the proxy setting and hits the pages directly! I can see this by following Squid's access log while loading up different pages on the client using various browsers.

Do I need to block direct access in order to get the browsers to actually use the proxy settings specified?

Very odd... Anyone got experience from this?

[rungekutta]

Just tried an iOS client. That also silently ignores the wpad.dat settings whether I set it to full auto-discovery or whether I give it the wpad.dat URL. Just hits the web pages direct irrespective.

[rungekutta]

Another test - added a block rule in the firewall for ports 80 and 443. Unfortunately that still didn't get Safari to use the proxy settings, it just fails to load the pages instead.

Chrome in the meanwhile continues to read and respect the proxy settings as configured.

[bartjsmit]

Run a trace on the interface that the iOS devices connect with to confirm that they are sending WPAD packets and to compare the traffic with a Chrome client. Interfaces, Diagnostics, Packet capture.

Wireshark is your friend https://www.wireshark.org/

Bart...

[rungekutta]

Run a trace on the interface that the iOS devices connect with to confirm that they are sending WPAD packets and to compare the traffic with a Chrome client. Interfaces, Diagnostics, Packet capture.
Wireshark is your friend https://www.wireshark.org/

Thanks. How do I do that in practise? Do I need to install on OPNSense and run from shell or are there standard tools in the GUI somewhere to run the capture (later to be downloaded and analysed in Wireshark)?

[fabian]

In OPNsense you can use tcpdump (available as packet capture in the GUI) and open the file on a local Wireshark.

[rungekutta]

Ok guys, thanks for that tip, will come in handy. I managed to figure this one out without it though. It was kind of in the instructions... but not to its full implication, so to speak.

I cleared out browser caches to do some more debugging on access to wpad.dat and realised I had SSL redirection enabled (80 -> 443) on the OPNSense web gui. And it runs the default self-signed certificate - I had taught my browser to trust it so didn't notice at first. But the invalid certificate is why I got erratic and different behaviour from different clients and browsers with respect to loading and parsing wpad.dat.

So it left me with two options; either get a valid certificate for OPNSense's LAN web gui, or switch the whole GUI to port 80 and http, which is kind of terrible but the lazy option. It's a shame that OPNSense won't let me serve out wpad.dat on http/80 and run the web gui on https/443 at the same time.

In any case, I got it to work reliably now across all the clients I've tried including Mac and Windows.

[hbc]

Hi,

you can serve wpad.dat on port 80 with gui on other port. You just have to use nginx as webserver, bind it to localhost port 80, set your webroot to /usr/local/www and add a location match = '/wpad.dat' (else all web gui file = gui will be accessible).

Do not forget a firewall rule that allows your lan to connect to 'This Firewall' port 80.

What you have to care for:
in dhcp service do not use WPAD checkbox, but add option your own (the box will create URL with your web gui port)
in unbound do not use WPAD checkbox, but add overwrites your own (checkbox will create correct A/AAAA entries, but the TXT records with "service: wpad" will point to web gui port )

Since this question is asked so often, I think I will write a morer detailed faq/Howto.

I just added a HOWTO: https://forum.opnsense.org/index.php?topic=12026.0

[rungekutta]

Thanks. That makes a lot of sense.