Is there a doc about the OPNsense method of policy routing?

[whitwye]

Given that OPNsense isn't using multiple routing tables (which is how Linux is typically configured for policy routing), but instead is using PF's route-to and reply-to options, where can I learn about what in theory should be happening with those as interface availability changes?

I'm intrigued by Franco's statement that there's an optimal pf rule set that will make the result robust, but puzzled on what that rule set should look like. As I've mentioned in other threads, so far I can't get OPNsense to handle WAN2 correctly when WAN1 is taken down. I'll be thankful for any suggestions of recipes that should work, or pointers to documentation that gives enough background to deduce what such recipes should look like.

Specifically, what rules applied to either the floating or WAN2 interface rule set would enable WAN2 to successfully return or originate traffic, regardless of WAN1's state? I see no evidence that OPNsense will originate traffic on WAN2 ever. But it at least returns traffic on WAN2 while WAN1 is up, yet fails to return traffic on WAN2 once WAN1 is down -- an odd and unexpected dependency. Taking WAN1 down removes the default route from the system; but apparently the power of pf route-to and reply-to rules should make success independent of that. Besides, the WAN1 default being their or not shouldn't on the face of it affect the success of WAN2 in replying on its IPs, since this is working with WAN1 up, where the reply doesn't take that default route anyway.

[franco]

Hi whitwye,

The guide here explains the preferred setup(s):

https://docs.opnsense.org/manual/how-tos/multiwan.html

Step 3 and 5 are what mostly create issues in our experience.


Cheers,
Franco

[whitwye]

Franco,

I've read that page dozens of times. Is there no other documentation on this?

Step 3 I followed, and am certain it's right. It's simple enough.

As for step 5, the example there is for the LAN IP of the firewall and DNS service. We're not running DNS on the firewall. We're not concerned (yet) with traffic behind the firewall being sent out correctly either. Is there an (undocumented) requirement that the firewall be used as a DNS server for MulitiWAN to work?

RIght now, if we take WAN1's interface down, incoming traffic from outside on WAN2 is no longer returned by WAN2, even though it is returned by WAN2 just fine if WAN1 is also up. And traffic generated from the firewall does not find its way out WAN2.

Is there documentation pertinent to those problems, or the theory by which they are supposed to be handled, or steps to diagnose them?

Thanks,
Whit