OPNsense Forum

English Forums => General Discussion => Topic started by: CanadaGuy on February 20, 2023, 03:45:50 pm

Title: reply to incoming packet not respecting policy based routing
Post by: CanadaGuy on February 20, 2023, 03:45:50 pm
I'm new to OPNsense and I'm coming from Ubiquiti EdgeOS (similar to old Vyatta). I have policy based routing configured and working for the most part. Connections initiated from my network go out the correct interface (WAN2 - remoteips_server alias). I have NOT configured this as a traditional wan interface, I just wanted to route some traffic through this interface (WireGuard tunnel on a server) instead of the main WAN interface. All other aspects of the network are working as expected (gateways, vlans, firewalls, etc.)

When I initiate an external connection that comes in from WAN2 (remoteips_server alias) to a host, the reply gets routed out WAN1 instead. My initial guess is that the incoming connection uses an existing firewall state to get routed back out the default gateway (WAN1). An alternative is that there are default rules on all interfaces which pass related connections before processing the rest of the rules where my PBR is specified.

How do I get a reply to an incoming connection to either go out the WAN it came in, OR add a firewall rule that will operate on new AND existing connections. My intention is that all internet traffic should go out remoteips_server gateway, except traffic defined in prior rules like local LAN connections.
Title: Re: reply to incoming packet not respecting policy based routing
Post by: zan on February 21, 2023, 04:09:03 am
Set the 'reply-to' of the pass rule in the wg interface the packet came through to the wg gateway.
Title: Re: reply to incoming packet not respecting policy based routing
Post by: CanadaGuy on February 21, 2023, 06:13:28 pm
I have looked all over for a "reply-to" field and have no managed to find it...and I finally found it but it doesn't seem to impact it. I see this: "packets on WAN type interfaces reply to their connected gateway". How do I specify an interface as a "WAN type" interface? Just by specifying an upstream gateway?

I have a server that WireGuard terminates on as I would like to keep this functionality separate from the firewall so that the encryption processes won't consume the firewall CPU. This server is on VLAN 150, and I added the reply-to gateway on the VLAN 150 in ruleset. Each server is on an isolated VLAN 240 (or other) and the traffic goes there.

Does it need to be more specific to the interface? Will it only work if the WireGuard interface is directly in the firewall? The reply seems to get passed as an established connection before reaching this firewall rule.
Title: Re: reply to incoming packet not respecting policy based routing
Post by: CanadaGuy on February 21, 2023, 06:49:32 pm
it appears I was doing only one step or another, but not the two required steps together. I needed to configure my WireGuard server AS an upstream gateway, then I configured the WireGuard server VLAN to USE this upstream gateway and it seems to be working.

Does this all seem reasonable?
Title: Re: reply to incoming packet not respecting policy based routing
Post by: CanadaGuy on February 22, 2023, 08:01:09 pm
I finally have closure as I found my real issue which were some floating "pass" rules that meant the rule with my reply-to configuration was never hit.

I noticed that I couldn't edit the floating rule, so I couldn't see when I added it. Is that expected?