Struggling with OPNsense in a DMZ behind ISP router

[Untoasted9563]

Hi all,

Recently I switched to fiber and got a new router/modem from my ISP. That damn thing does not have a bridge mode and on top of that the subnet cannot be changed (192.168.1.0/24). Unfortunately, I have a contract period of two years, and it looks like I have to accept that.

Like many of those routers, it offers a DMZ feature, which as I understand is just port forwarding of everything to the DMZ host and placing the host in a /32 network for isolation (192.168.1.254/32 in my case). Generally, this worked, but I have some issues with that.

A bit of background info:
- I run HAproxy for different self-hosted services (originally based on the Hellsite tutorial: https://forum.opnsense.org/index.php?topic=23339.0)
- As per that tutorial, I created a virtual IP (IP Alias 127.4.4.3/32 on Loopback interface), which is set as a listen address on the public server in HAproxy. Alongside the SNI listening on 0.0.0.0:443
- the services are in different DMZ vlans
- i have an MGMT network for Ubiquity gear and some other admin webUIs. Before, this was 192.168.1.0/24, which i moved to 192.168.5.0/24 in order to prevent overlap with the ISP subnet.
- I have a WAN firewall rule, that allows IPv4 TCP 443/80 to "WAN address", which was allowing for remote access on HAproxy, when OPNsense was having a public IP at the WAN interface (remember, now it has 192.168.1.254).
- Gateway for 192.168.1.1 was created (Upstream and Far checked) and specified in the WAN interface.

Now the strange thing happens, when I am trying to connect to one of my services from remote:
I see the attempt being routed to 192.168.5.254 and therefore be default-denied. What is this IP? I have never typed it anywhere. Of course it could be a typo, but the only place, where I have typed 1.254 is the static IP at the WAN interface, and this is correct (otherwise the external connection attempt would not even be registered).

Since it is not "WAN address", it does not trigger the existing rule.
Trying to circumvent this, I added the 192.168.5.254 IP alongside "WAN Address", but HAproxy is not catching those requests and lists nothing in the log.


Another issue (maybe related, thats why I shortly mention it here) is the lack of internet access in the MGMT network, and that network alone. All other networks and VLANs under OPNsense have internet access.
ping -S 192.168.10.1 8.8.8.8  is successful
ping -S 192.168.5.1 8.8.8.8  fails.
Despite having a temporary allow all protocols from any to any on all ports, in/out as top rule in MGMT rules.


Can anybody make some sense of this? did I forget anything due to the fact that OPNsense has a private IP on the WAN?

Thankful for all help and pointers
Cheers,
Untoasted

[coffeecup25]

First, apologies, but I didn't read all that. I admire anyone who can follow someone else's description of their network. Everyone has a different special situation, it seems. I've never been able to.

My first instinct was to use 'problem decomposition'. Basically, get it working as you like in the simplest form possible, without any of the extras included. Then layer the other features one at a time. Like a jigsaw puzzle. Nobody pours a jigsaw puzzle out of the box to see it automatically put itself together. Start with basic internet that works reliably and branch out from that.

Good luck.

[pfry]

[...]My first instinct was to use 'problem decomposition'.[...]

Perhaps "System: Configuration: Backups" -> "Downloads", download and search for ".5." or similar. But it doesn't sound like it'll be there, as hitting the default deny suggests an external source.

[...]ping -S 192.168.10.1 8.8.8.8  is successful[...]

Is it? Typo, or another subnet?

[coffeecup25]

[...]My first instinct was to use 'problem decomposition'.[...]

Perhaps "System: Configuration: Backups" -> "Downloads", download and search for ".5." or similar. But it doesn't sound like it'll be there, as hitting the default deny suggests an external source.

[...]ping -S 192.168.10.1 8.8.8.8  is successful[...]

Is it? Typo, or another subnet?

Good idea.

[Untoasted9563]

Hi guys, thanks for taking the time to read through it (at least in part ;) ) and answering me.

The downloading config thing is indeed a good idea and could have probably also brought me closer to the solution.

With help of chatGPT I was zeroing in to NAT. And I found a one-to-one NAT entry, which I couldnt explain. Maybe creating a gateway and setting it on the WAN caused this? Maybe a remnant from before the network switch? No idea. This entry was NATing 192.168.1.1/24 to 192.168.5.1 and was the culprit for both of my problems (creating the weird 192.168.5.254 IP and screwing up my outbound traffic from ..5.0 subnet).

Anyways, thanks again.