Multi Wan issue, LAN losing Firewall access when specific WAN is selected

[mellow65]

Greetings all, semi long PFsense user (2 years now), almost as long trying to convert to OPNsense (year+-).

UPDATE#2: Well, while my "fix" allowing access to the firewall worked to get access to the firewall, I found that I can't ping anything on other networks.  Which makes sense to some level, but doesn't fix the underlying problem of dropping firewall access when a specific wan is selected.

Update:  It wasn't specifically DNS being blocked, OPNsense no longer passes data to the firewall once you change your firewall from Default to a specific WAN. I found my answer to it, but it still doesn't seem correct that it should be doing that.


The sweet irony of all this as I was writing this, I figured out what was happening, and it was DNS.  It’s always DNS!!    But I figured I would still post this because maybe someone else may find this useful. 

Computer: HP T730 with 4 port intel nic

End goal, have two separate gateways (GW) for me to decide which clients goes over which GW based on rules in the firewall>rule>lan section.  This is currently how my PFsense is set up, it's not based on load balancing or fail over.  Just two separate gateways, because...reasons.

Started with a factory reset box and all updates, shut down IPV6 where I could (not against it, just don’t understand it enough, and I wanted to take that off the plate as a source of the problem) Set default DNS to 8.8.8.8 and 8.8.4.4 in wizard.

Set up OpenDNS via Unbound DNS with DNS over TLS.

I made two WAN interfaces with static IPV4s, WAN1, 192.168.15.5 and WAN2, 192.168.8.5. These are both open IPs on my modems.

Set up two GWs, each using the designated gateway IP, 192.168.15.1, and 192.168.8.1, both the GW are on their own internet source.

Went back to WAN1 and WAN2 interface and picked the appropriate IPv4 Upstream Gateway.

Manually set up NAT rules for WAN1 and WAN2, copying what was originally created when the wizard was complete.  Four NAT outbound rules in total.

Only have the single LAN and one rule, pass the internet.  My computer is plugged directly into the port that is specified in the interface for LAN.

Here’s where my issues starts,

If that lone LAN rule has GW set to default, internet works just fine, as soon as I manually pick either WAN1 or WAN2 specifically, the internet dies.  I can still ping 8.8.8.8, 8.8.4.4, and 1.1.1.1, which lead me to thinking it was a DNS problem. 

When the LAN rule is set to Default GW, I could change what internet source my traffic was going out of if I set WAN1 and WAN2 both to be in the default pool (Upstream Gateway), and change around the priority.

My question is, what is happening to my DNS when I switch to a specific GW?

I can fix it by putting a DNS rule at the top of my rules.

With this rule, I can freely switch between the two WANs and it works as I want it to, I’m just trying to figure out if there’s a more elegant way of doing this VS putting a DNS rule at the top of every new interface.

Thanks for any insight!!!

[mellow65]

Plot thickener, so when I set the WAN to a specific WAN (WAN1 or WAN2), It doesn't just kill DNS, I can't ping the interface IP at all.  Yet I can still SSH into it on that same IP address. 

[mellow65]

As I work through my issues related to trying to do some Multi WAN routing, I think I've come up with the least intrusive answer to my issue.  By adding a new rule setting the source to LAN Net and Destination This Firewall, this is giving me what I'm looking for. 

This allows pinging of all interfaces and all ports so my DNS works so the internet works!!! 

I still don't fully understand why the LAN network loses access to the firewall when I switch from the default WAN to a specific WAN interface.  So if anyone has any insight into why this is happening, I would love to hear why this is and if there is a more elegant solution than what I have here.