Messages

Thank you for the tutorial. I use Exchange 365 and am wondering what the use cases for this are?

Thanks for the additional thinking points.

Frankly I didn't clock this could be a DNS issue as I interpreted the 503 error rather than the request was received by a service (expected HAProxy) but misconfigured where to forward to).

I did configure pihole to use unbound, which is why I didnt think about it inititially. Pihole has upstream DNS pointing to Google DNS, enabled 'Never forward non-FQDN A and AAAA queries', 'Never forward reverse lookups for private IP ranges' and 'Use Conditional Forwarding'. I don't see where I can point it back to Unbound (would I configure Unbound as the Upstream DNS?)

Also you never mentioned anything about pi-hole in your previous posts so for the future I'll pass on a great tip from the Zen of Python:

Explicit is better than implicit

Yeah, sorry for that. I thought the same when the netstat said "pihole"... (mea culpa). thanks a lot for the support and responsiveness so far, appreciate it.

Great tip - tracking with nslookup what happens I can see that my pihole catches the DNS requests:

Code Select Expand

C:\>ipconfig /flushdns
Windows IP Configuration
Successfully flushed the DNS Resolver Cache.

C:\>nslookup opnsense.mike0000.dynprovider.com
Server:  pi.hole
Address:  192.168.5.95

Non-authoritative answer:
Name:    mike0000.dynprovider.com
Address:  [WAN IP ADDRESS]
Aliases:  opnsense.mike0000.dynprovider.com


C:\>nslookup mike0000.dynprovider.com
Server:  pi.hole
Address:  192.168.5.95

Non-authoritative answer:
Name:    mike0000.dynprovider.com
Address:  [WAN IP ADDRESS]

Now I added a DNS record in pihole pointing opnsense.mike0000.dynprovider.com to 192.168.5.1

Code Select Expand

C:\>nslookup mike0000.dynprovider.com
Server:  pi.hole
Address:  192.168.5.95

Non-authoritative answer:
Name:    mike0000.dynprovider.com
Address:  [WAN IP ADDRESS]


C:\>nslookup opnsense.mike0000.dynprovider.com
Server:  pi.hole
Address:  192.168.5.95

Name:    opnsense.mike0000.dynprovider.com
Address:  192.168.5.1

I can now access the web GUI of the OPNsense box on "opnsense.mike0000.dynprovider.com:XX443" via the browser over WIFI. This did not work before.

Testing via a different machine (using a SIM card to go online from outside of the LAN) and tunneling in via OpenVPN I cannot access the page via the same URL yet. Checking nslookup again reveals that pihole is not used for DNS, rather a 172.xx private IP:

Code Select Expand

C:\>ipconfig /flushdns
Windows IP Configuration
Successfully flushed the DNS Resolver Cache.

C:\>nslookup opnsense.mike0000.dynprovider.com
Server:  UnKnown
Address:  172.20.10.1

Non-authoritative answer:
Name:    mike0000.dynprovider.com
Address:  [WAN IP ADDRESS]
Aliases:  opnsense.mike0000.dynprovider.com


C:\>nslookup mike0000.dynprovider.com
Server:  UnKnown
Address:  172.20.10.1

Non-authoritative answer:
Name:    mike0000.dynprovider.com
Address:  [WAN IP ADDRESS]

It seems the last bit I need now is to make OpenVPN clients use the pihole DNS resolver on 192.168.5.95?

Is this a workaround / dirty way to do this, i.e. should I be fixing this via HAProxy somehow (potentially missing front/back end)?

Does that DNS record resolve to your public IP?

Checking with ping and tracert the dynamic dns name (mike0000.dynprovider.com) resolves to my WAN IP. Same for a subdomain (opnsense.mike0000....)

If so you need to activate NAT loopbackb or create a DNS override for the same hostname that resolves to an internal IP where webui (or HAProxy) is listening.

How can I create the DNS override to resolve that hostname to an internal IP? I went to Services --> Unbound DNS --> Overrides and added an entry under both Host Overrides as well as Domain Overrides pointing at the IP, but still does not work (after restarting the service).

Any idea or specific pointer? Thanks so far!

Hi Sorano, thanks a lot for your fast replies and help!

I tried killing HAProxy but got errors - I believe the user I configured does not have sudo rights (something to be fixed next).

What I did now, that helped me to regain access is log in locally (plug in monitor and keyboard into the box) and via single user mode mounted the root drive, and edited the port in the config.xml directly.

I'm back in now and can connect via local LAN and VPN tunnel to the web GUI on IP:xx443.

Next step would be to fix the issue that I cant access the GUI via the subdomain of the dynamic DNS provider I configured, i.e. https://opnsense.mike0000.dynprovider.com:XX443 via VPN.

Edit: typo

[Any tips of how to regain access to make the change?]

I am finally back home now with physical access to the box. I am at a loss though of how to log back in and make the change to the config of the HTTPS port back to the earlier xx443.

I tried connecting via the specific admin LAN port I configured, but HAProxy applies same rules there. I also tried connecting via SSH but my user, whilst an admin, does not have root privileges and cannot edit the config.xml (that would be the easiest...).

Currently a site access is captured by HAProxy and results in the 503 error.

Any tips of how to regain access to make the change?

Thanks sorano. I have edited the OP above to clarify (the 99443 port was an example, I didn't put the actual port I am using here).

I have further tested and wanted to see if I could remote access the GUI when the port is set to 443. Now I obtain the 503 error as shown above (when connected via VPN both when accessing the URL as well as the LAN IP 192.168.1.1:443. The prior 192.168.1.1:XX443 is no longer available (of course, I changed it) and times out. So looks like I am locked out for now and have to physically access it when back home.

What is weird is that my NAS is also accessed through port 8080 on 192.168.1.60:8080, but listens on 443 as well. Going to 192.168.1.60:443 also gets me to the NAS' web GUI, whereas I expected a 503. I'm confused

Meanwhile would be great to get input/ideas here - I strongly suspect this is related to the HA Proxy that I am running. The original setup/config was done following this tutorial: https://forum.opnsense.org/index.php?topic=23339.0

I love OPNsense, thanks for the effort going into this by the developers and the community.

Just sent $100 to support this work. I'm considering a subscription.

Regards
Mike

P.S. if anybody could help with my question that would be great!

[Background/status:]

Hello,

I've got OPNsense set up and running very well for half a year or so, OpenVPN included.

Background/status:

• Access to the admin interface is https only (HTTP Strict Transport Security enabled) and via a modified port (192.168.1.1:XX443)
• The OPNsense box is configured with Hostname opnsense and Domain mike0000.dynprovider.copm
• I have set up a dyn dns provider with my public static IP (let's say mike0000.dynprovider.com)
• I have set up and issued SSL/TLS certificates via Let's Encrypt (automated) successfully for this domain
• I've got a QNAP NAS box that has a web interface accessed via HTTPS. I can access it via internal LAN or through VPN via the local IP 192.168.1.60:8080 but not via https://nas.mike0000.dynprovicer.com:8080
• HA Proxy is running on the OPNsense box and I suspect this could also be the culprit, meaning it handles the http(s) traffic and doesnt let the request through to the right host (the OPNsense box). The HAProxy_ports list is 80 and 443. The original setup/config was done following this tutorial: https://forum.opnsense.org/index.php?topic=23339.0

Goal:
Access via the browser https://opnsense.mike0000.dynprovider.com:XX443 and be able to see and log in to the OPNsense admin web GUI. This should only be possible from a host that is connected via OpenVPN, which is configered to be the address space 10.0.7.0/24). Otherwise all inbound traffic should be rejected on not be able to access mike0000.dynprovider.com or any subdomains on any ports (I don't want to break things and have no internet though).


Issue:

At the moment the following is happening when accessing via a browser:

• Access https://mike0000.dynprovider.com (from a remote machine, with and without VPN) results in a 503 HTTP error ("503 Service Unavailable. No server is available to handle this request.")
• Access https://mike0000.dynprovider.com:XX443 (from a remote machine, with and without VPN) results in a timeout ("Safari couldn't open the page because the server stopped responding.")
• Access https://opnsense.mike0000.dynprovider.com (from a remote machine, with and without VPN) results in a 503 HTTP error ("503 Service Unavailable. No server is available to handle this request.")
• Access https://opnsense.mike0000.dynprovider.com:XX443 (from a remote machine, with and without VPN) results in a timeout ("Safari couldn't open the page because the server stopped responding.")

In case 1. and 2. above I want to prevent all access.
In case 3. above I want to prevent all access.
In case 4. above I want to allow access for LAN or VPN users (i.e. hosts with an IP from the 10.0.7.0/24 subnet).
I don't want LAN users (i.e. hosts with an IP from the 192.168.1.0/24 subnet) to have any restrictions.

Screenshot of current firewall rules:
Attached

[edit]
I suspect I need a WAN rule rule to say "allow inbound TCP/UDP traffic from subnet 10.0.7.0/24 to host opnsense on port XX443".
Looking at behaviour on point 1. and 3. above I think
[/edit]

In the meanwhile if I missed (despite searching for it) a tutorial that explains what to do I'm grateful if someone could point me to it.

Thank you
Mike

Edit: amended to make clearer and to read and added further clarifications.   

Edit 2: I have the strong suspicion that HAProxy may be relevant here and potentially needs some configuration for the local subdomains map rule and public subdomains map rule. The NAS' web interface is accessible on port 8080 (as mentioned above) and also on 443 via it's IP but also not publicly via the URL (as stated above; this is a behaviour I would only want to allow through VPN).

The weird thing is that the NAS is accessible on 8080 and 443 (NAS config allows access on both) without 503 error, but the OPNsense box throws a 503 error now that it is on port 443. Weird

Edit 17/7/23: updated subject to be more accurate as previous was misleading

TheHellSite, thanks a lot for all the work that you've put into this tutorial. I have followed every step of it and almost everything is working well.

One issue I am facing is that when I ping a local domain (e.g. opnsense.mydomain.com (router/fw box), nas.mydomain.com (qnap nas)) the IP gets resolved as my external WAN IP address.

When I direct my browser at one of my internal domains I'm not getting the same result as when I simply go to the corresponding IP.

I am not sure whether there is a misconfiguration in the HAProxy setup, or whether it is in fact unrelated to your tutorial. Below are the outputs I see from accessing two different subdomains from both my phone on cell service, as well as from internal wifi:

When accessing my OPNSense webui on opnsense.mydomain.com:55443 (internal IP 192.168.5.1):

• Accessing via browser (Firefox/Chrome/Safari (iPhone) I'm getting the following error: "The connection has timed out. An error occurred during a connection to opnsense.mydomain.com:55443" (Firefox); "This site can't be reached. opnsense.mydomain.com took too long to respond." (Chrome); "Safari could not open the page because the server stopped responding." (Safari iPhone)

When accessing my QNAP NAS on nas.mydomain.com (internal IP 192.168.5.60):

• Accessing via browser (Firefox/Chrome/Safari (iPhone) I'm getting the following error: "503 Service Unavailable. No server is available to handle this request." This suggests to me the http server is actually responding to the request but not serving an SSL page, even though https://192.168.5.60 returns the NAS login page.

• Testing the server response from shell on the nas [edit1]:

Code Select Expand

user@OPNsense:~ $ wget --save-headers http://nas.mydomain.com:8080
--2023-01-08 12:47:22--  http://nas.mydomain.com:8080/
Resolving nas.mydomain.com (nas.mydomain.com)... 192.168.5.60
Connecting to nas.mydomain.com (nas.mydomain.com)|192.168.5.60|:8080... connected.
HTTP request sent, awaiting response... 200 OK
Length: 580 [text/html]
Saving to: 'index.html'
index.html 100%[=====================================================================================>] 580  --.-KB/s    in 0s
2023-01-08 12:47:22 (139 MB/s) - 'index.html' saved [580/580]

When checking the response on https request an error comes (with the '--no-check-certificate' parameter yields the same result/output as the http request):

Code Select Expand

user@OPNsense:~ $ wget --save-headers https://nas.mydomain.com
--2023-01-08 13:01:21--  https://nas.mydomain.com/
Resolving nas.mydomain.com (nas.mydomain.com)... 192.168.5.60
Connecting to nas.mydomain.com (nas.mydomain.com)|192.168.5.60|:443... connected.
ERROR: cannot verify nas.mydomain.com's certificate, issued by 'CN=R3,O=Let\'s Encrypt,C=US':
  Unable to locally verify the issuer's authority.
To connect to nas.mydomain.com insecurely, use `--no-check-certificate'.


There seem to be two issues here: 1) there is a certificate error (I imported the acme/LE wildcard .crt and .key into the NAS).

But 2) more generally as observed with the pings thrown at the WAN IP rather than the correct internal IP the request from the browser is also being forwarded to the WAN (mydomain.com) - this is shown by wget on mydomain.com which also returns a 503, the same that the browser does:

Code Select Expand

user@OPNsense:~ $ wget --save-headers https://mydomain.com
--2023-01-08 13:13:59--  https://mydomain.com/
Resolving mydomain.com (mydomain.com)... 185.176.xxx.xxx [WAN IP]
Connecting to mydomain.com (mydomain.com)|185.176.xxx.xxx [WAN IP]|:443... connected.
HTTP request sent, awaiting response... 503 Service Unavailable
2023-01-08 13:13:59 ERROR 503: Service Unavailable.

Any ideas or can you recommend any tools to do further troubleshooting or does anyone spot what the issue is?

The purpose of my setup is that all subdomains should only be accessible from the LAN or through VPN (this is set up correctly, I can VPN in via OpenVPN).

-------------------------

edit1: inserted code view of wget header response for nas.mydomain.com access - seems to be working in shell but not from browser

-------------------------
Many thanks
Michael

Attached the config files requested
- HAProxy Config Export
- HAProxy errors and/or log entries
- Details about setup: above, but happy to elaborate further if unclear