OPNsense Forum

English Forums => 18.7 Legacy Series => Topic started by: drivera on October 30, 2018, 03:16:24 am

Title: Multi-WAN not routing incoming packets back whence they came properly
Post by: drivera on October 30, 2018, 03:16:24 am
Hi!

I have a Multi-WAN setup, which after some toil (mostly due to my newbness :D ) appears to be (mostly) working the way I want it to (thanks to mimugmail for helping me out!). However, there's one thing not working right now that I can't see my way past.

The WANs are set up for a failover scenario: if the primary fails, the secondary takes over. This works well enough. The problem is that while everything is up (i.e. primary is up), I'm unable to ping the secondary interface from a remote location. Pinging the primary works just fine.  When the primary is down and the secondary is up, then I can ping the secondary (now primary due to the failover) just fine from that same location.

The issue, I believe, has to do with default gateways. I can only set up one default gateway. I had to enable gateway switching to get around other problems (discussed here (https://forum.opnsense.org/index.php?topic=10085.0), there's some more fun hijinks on that topic but I digress).


Using the Packet Capture utility I can see that the traffic does arrive fine to the firewall on the secondary while the primary is up.  The problem is that a response is never sent out. This is because the primary had to be set as the default gateway (see the above link) for gateway switching to work, so the O/S (apparently) doesn't know to give those packets special treatment and bounce them right back the network interface they came from.


I know OPNSense isn't Linux, but the way to solve this in Linuxland would be to have a routing rule (using ip route) specifying that packets originating from a given interface's address are to be routed using a special routing table (built for that interface) where the default gateway is that interface's.


I have no clue how to do that on OPNSense-land (*BSD-land)...


Can you guys help me out?
Title: Re: Multi-WAN not routing incoming packets back whence they came properly
Post by: x2416 on October 31, 2018, 07:55:01 pm
Go into each interface, and at the bottom where you can select a gateway, select the correct gateway.

Also, check out Firewall -> Settings -> Advanced and check Sticky Connections under Multi-WAN :-D
Title: Re: Multi-WAN not routing incoming packets back whence they came properly
Post by: drivera on November 01, 2018, 04:04:22 am
Sticky connections was already enabled, and per-interface gateway selection appears to only be possible for statically-configured interfaces (none of mine are - all are DHCP).

Any other ideas?
Title: Re: Multi-WAN not routing incoming packets back whence they came properly
Post by: mimugmail on November 01, 2018, 08:00:17 am
Screenshot of Firewall : Settings : Advanced please ...
Title: Re: Multi-WAN not routing incoming packets back whence they came properly
Post by: drivera on November 01, 2018, 02:58:08 pm
Here you go... anything else I can provide you to help diagnose?
Title: Re: Multi-WAN not routing incoming packets back whence they came properly
Post by: mimugmail on November 01, 2018, 04:38:30 pm
Kill states and Disable Force Gateway is enabled on my side.
Title: Re: Multi-WAN not routing incoming packets back whence they came properly
Post by: drivera on November 01, 2018, 08:01:38 pm
Enabling either (or both) setting(s) had no effect. Even rebooting once they were enabled. Any other thoughts?
Title: Re: Multi-WAN not routing incoming packets back whence they came properly
Post by: mimugmail on November 01, 2018, 09:04:59 pm
I'll check tomorrow, have a customer with similar setup, also with DNAT for both WAN IPs to same host, so it must ne working anyhow :)
Title: Re: Multi-WAN not routing incoming packets back whence they came properly
Post by: drivera on November 01, 2018, 09:06:01 pm
I'm sure it's some little detail somewhere. I can send you a (sanitized) copy of my configuration, if that would help.

Cheers!