OPNsense Forum

Problem

The default OPNsense auto proxy configuration is designed to work best with plain http (port 80). As soon as you use port redirect to https (443) you will run into problems, since

• some auto proxy configurations mechanism rely on http
• you may run into certificate issues (self-signed) with https

The best option is to provide wpad.dat via http, so you can even restrict your web gui port just to your admin pcs.

How to configure

• redirect your web gui to https or other port
• restrict this port to your admin pcs
• install nginx web server

Configure nginx

• Nginx: Configuration --> HTTP(S) --> Location




• Description: WPAD
• URL pattern: wpad.dat
• Match type: Excact match("=")
• File system root: /usr/local/www


• Nginx: Configuration --> HTTP(S) --> HTTP server
  
  
  
  
  • Listen port: 80
  • Server name : localhost
  • Location: WPAD
  • File system root: /usr/local/www
  
  


• Nginx: Configuration --> General settings




• Enable nginx: enabled

Now nginx will listen on port 80 and provide the original wpad.dat file created via gui.

DHCP and Unbound provide gui options to enable WPAD support, but these options will (partitual) create configuration entries that point to your (inaccessible) web gui port. So we have to add those option manually instead.

Configure DHCP service

• Services: DHCPv4: [Interface]
  
  
  
  
  • WPAD: unchecked
  • Additional Options:
    
    
    
    • Number: 252
    • Type: text
    • Value: http://[interface-ip]/wpad.dat

Configure Unbound
The A and AAAA records would already be right with the WPAD option, since these records cannot provide ports anyway, but there are also TXT records with port entries created that would be wrong. So we skip TXT records (not supported via gui) and just add A and AAAA records.

• Services: Unbound DNS: General
  
  
  
  
  • Advanced: Show advanced options
  • WPAD Records: unchecked
    
    
• Services: Unbound DNS: Overrides
  
  
  
  
  • Host: wpad
  • Domain: your [interface-domain]
  • Type: A or AAAA
  • IP: [interface ip]

Configure firewall
Add a rule that allows [interface net]:1024-65535 --> This firewall:80

With this configuration clients should be able to acquire either via DNS or DHCP a valid proxy configuration via port http (80).

you can move the file system root from the HTTP server to the location to make the rest of the web interface not available

You mean /usr/local/www/wpad.dat as root? One single file? That works? I thought the location match as restriction. As you mentioned in one of my other posts, it should create 404s for other resources.

no the normal www directory. the root is a directory from which the normalized path (the /wpad.dat) is relative.  For example:

Root: /usr/local/www
URL: http://localhost/wpad.dat (http://localhost/wpad.dat)
Result: /usr/local/www/wpad.at
The root can be set in locations and in http servers in the plugin.
If it is set up for the location only, the main server is not affected and you can point it to an empty directory (or leave the default) so it won't deliver anything else. If your location does not set a root, it will use the one from the server. I hope this helps to understand the internals of nginx a bit.

Ok, I see. There is another file system root option in location. I added it and fixed the connection server to location. When writing howto I missed that the WPAD location has to be chosen in server setting.

No problem, since the plugin is quite big, so it is simple to oversee a config option.
Sometimes even I have to search for them.

One related issue that I found is that if you disable redirection from 80->443 you lose the ability to load wpad from HTTP. See https://github.com/opnsense/core/issues/3416

Quote from: ssbarnea on April 14, 2019, 04:39:22 PM
One related issue that I found is that if you disable redirection from 80->443 you lose the ability to load wpad from HTTP. See https://github.com/opnsense/core/issues/3416

For this reason, I wrote this HOWTO. No matter what port you use for webgui, WPAD will be provided via HTTP.

Hi , Firewall rule does not apply , only 1 - 65535, What would be the correct rule?

Quote from: rickygm on May 20, 2019, 10:52:42 PM
Hi , Firewall rule does not apply , only 1 - 65535, What would be the correct rule?

The rule is correct. A range from 1024:65535 should work for source ports. But you can also just use "any"

Quote from: hbc on May 21, 2019, 07:01:54 AM

Quote from: rickygm on May 20, 2019, 10:52:42 PM
Hi , Firewall rule does not apply , only 1 - 65535, What would be the correct rule?

The rule is correct. A range from 1024:65535 should work for source ports. But you can also just use "any"

Hi , I made the change in the firewall rule, but I can not test the wpad, nginx does not start I see the log and it says that port 80 is in usao, I think it's because of the gui

[emerg] 60602#100295: bind() to [::]:80 failed (48: Address already in use)
2019/05/21 17:02:51 [emerg] 60602#100295: still could not bind()

root     lighttpd   49720 5  tcp4   *:444                 *:*
root     lighttpd   49720 6  tcp6   *:444                 *:*
root     lighttpd   49720 7  tcp4   *:80                  *:*
root     lighttpd   49720 8  tcp6   *:80                  *:*

there will be some way to change port 80 of the gui?

in the administation page in the system section

I already did, but still using port 80

you need to disable the web ui redirect rule.

excelente , later need reboot de OPNsense , work gr8

I followed all the steps of the howto, but wpad does not work ,some form of debug I can do, to see where the error may be

any help?

What happens if you type in browser "http://firewall-ip/wpad.dat"? Does it download wpad.dat?

Can you telnet to firewall-ip port 80? Port open? Did you create a wpad.dat file via webproxy menu? Does it exist as /usr/local/www/wpad.dat?

What say log files. You have to analyze more analytic. Just saying it dos not work, it not much help.

Hi , I did the three different tests:

1 - telnet ip lan firewall port 80 successful

2 -  download wpad.dat successful

3 - the file exists in the path, but the configuration inside does not show anything

I did a packet inspection with wireshark and I see that the dhcp is not serving the wpad.

I attach images.

Quote3 - the file exists in the path, but the configuration inside does not show anything

Did you create matches, proxies and rules for wpad in webproxy configuration? Also very important: the wpad.dat is only written to disk, when you click save in any other webproxy tab.
Maybe they should add an apply button to pac tab, to just write wpad.dat to disk.

Not every system supports dhcp. It's also important to as the wpad dns entries. Do a tcpdump on port 53/udp and check requests for hostnames starting with wpad

it's strange, but I can not get it to surf, I see that the dhcp is serving the wpad and the dns too, take a look at my screenshot.

I forgot to add, my configuration in the proxy

So wpad is delivered, but proxy not used? Seems to be wrong logic in wpad.dat for proxy selection.

I would check 3 things:

• Browsers have auto proxy detection enabled
• nginx access log for download access of wpad.dat
• functionality of wpad.dat

Most browsers allow to set the url for proxy auto configuration statically. Enter your url to wpad.dat manually and check whether proxy is selected like you intended. If not, you have a wrong logic.

2: test it using curl http://wpad.example.com/wpad.dat
3: test you can test that using node.js (basic syntax test, in theory also if the file is working)

if those are fine, the PAC file may not be used.

Quote from: ssbarnea on April 14, 2019, 04:39:22 PM
One related issue that I found is that if you disable redirection from 80->443 you lose the ability to load wpad from HTTP. See https://github.com/opnsense/core/issues/3416

This issue could be solved by omitting https redirects for wpad.dat file.

To have the WeBGUI being redirected to port 443 and wpad.dat file being accessible via port 80 and port 443 simultaneously the redirect expression in the webgui configuration script /usr/local/etc/inc/plugins.inc.d/webgui.inc has to omit https redirects for wpad.dat file.

According to lighttpd mod_redirect documentation (https://redmine.lighttpd.net/projects/lighttpd/wiki/Docs_ModRedirect (https://redmine.lighttpd.net/projects/lighttpd/wiki/Docs_ModRedirect)) short-circuiting the redirect rules without triggering a redirect is possible by specifying a blank target.

Hence, the redirect expression in the webgui configuration script /usr/local/etc/inc/plugins.inc.d/webgui.inc should be amended as follows:

    /* add HTTP to HTTPS redirect */
    if (
        $config['system']['webgui']['protocol'] == 'https' &&
        !isset($config['system']['webgui']['disablehttpredirect'])
    ) {
        $redirectport = $lighty_port != "443" ? ":{$lighty_port}" : '';
        foreach ($listeners as $listener) {
            if (is_ipaddrv6($listener)) {
                $listener = "[{$listener}]";
            }
            $lighty_config .= <<<EOD

\$SERVER["socket"] == "{$listener}:80" {
    \$HTTP["host"] =~ "(.*)" {
        url.redirect = ( "^/wpad.dat" => "" ,
                         "^/(.*)" => "https://%1{$redirectport}/$1" )
    }
}


I would be more than happy if the developers would take this change into consideration.

Hi,

I've a similar issue.

I've the wpad.dat


admin@OPNsense:/tmp % cat /usr/local/www/wpad.dat
/*
  PAC file created via OPNsense
  To use this file you have to enter its URL into your browsers network settings.
*/
function FindProxyForURL(url, host) {




if (((!isPlainHostName(host)) && (!shExpMatch(host, "*.home.lan")))) {
return "PROXY 192.168.1.1:3128";
}

   // If no rule exists - use a direct connection
   return "DIRECT";
}


but


curl --header "X-Forwarded-For: other.net"  -I 192.168.1.1/wpad.dat
HTTP/1.1 301 Moved Permanently
Location: https://192.168.1.1/wpad.dat
Date: Sat, 13 Jun 2020 18:57:44 GMT
Server: OPNsense


failed.


$ dig -t txt  opnsense

; <<>> DiG 9.11.19-RedHat-9.11.19-1.fc31 <<>> -t txt opnsense
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 29865
;; flags: qr aa rd ra; QUERY: 1, ANSWER: 0, AUTHORITY: 0, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 4096
;; QUESTION SECTION:
;opnsense. IN TXT

;; Query time: 0 msec
;; SERVER: 192.168.1.1#53(192.168.1.1)
;; WHEN: Sa Jun 13 21:12:17 CEST 2020
;; MSG SIZE  rcvd: 37


also doesn't show any TXT entries. Even I've checked the WPAD boxes for DHCP, unbind and did reboot.

Firefox is configured for auto detection, I didn't use transparent proxy and HTTPS yet. I'm testing squid/C-ICAP/clamav combo basically where I faced this problem. Does it really work only in transparent mode using rules?

You need to deliver it via HTTP as well. In your case you get a 302 because you are redirected to the HTTPS port.

Quote from: fabian on June 13, 2020, 10:22:26 PM
You need to deliver it via HTTP as well. In your case you get a 302 because you are redirected to the HTTPS port.

thanks. Even after some investiagting I have to use the https proxy. The logs shows only my Fedora WLAN check (http://fedoraproject.org/static/hotspot.txt) is using HTTP :(

Does applying ACME LetsEncrypt certificate simplify the use of, e.g. transparent proxy, since no self-signed certs are required to be distributed?

They cannot be used for the proxy since you need a CA certificate