Messages

Added wrinkle:

I recently tried adding a WG S2S setup and inbound traffic from the remote site works fine. I can see ICMP packets traversing from a client on their site to a server on mine. I can also see the echo response coming into my main site's LAN port... and also going out the secondary WAN, NAT'ed, and destined for 192.168.2.2. It is supremely weird.

I just double checked the instructions about policy based routing (PBR) on multi WAN setups, and I am doing things as I expect I should: https://docs.opnsense.org/manual/how-tos/multiwan.html

At the top of the LAN firewall rules, I have PBR rule for any traffic destined for Cloudflare to bypass the LB_GW and use secondary because primary has sometimes-problematic misattributed geo-IP info. That def works, and I was always impressed with how simple it was to setup, behaving exactly as I'd expect.

I have a rule near the top for allowing 192.168.0.0/16 to 192.168.0.0/16 (I plan on having a number of remote sites) with a "process immediately" box checked, so it should stop processing on that firewall rule hit. There's no GW set so it should dump to the routing table. But it doesn't route to WG as it should.

Immediately after that is the any/any rule passing all over traffic to the LB_GW setup with failover. Obviously, I've tested that before, so I know it works.

Still scratching my head...

Hi all,

I'm a bit lost as to why this is. I've got a WG setup from some time ago that was working fine up until I switched to dual WAN setup (I happen to use a failover config, not that it should matter).

What's weird is that ever since I setup the dual WAN, all my WG client traffic goes straight out to the internet via that backup/secondary link. Like, I'm dialing in on primary WAN, and it's all going out secondary WAN, EVEN THE INTERNAL TRAFFIC. I could see it in a pcap, destination IP address 192.168.2.2... like, wut.

Does anyone have any pointers on what might be causing that? I have some networking background, followed the Deciso guide for dual WAN with failover, and am confident I wouldn't have intentionally setup/misconfigured it in such a way as to route WG over secondary or something.

It's boggling my mind and any pointers or perspectives are appreciated!

WAN1 is primary, WAN2 is secondary. Primary is used for all traffic unless monitoring stats aren't met (loss or latency), in which case it fails to secondary. Primary has 8x the bandwidth of secondary, hence the preference.

When I pull the upstream on primary (so its local L2 link stays online, but it cannot reach its gateway or internet), it fails to the secondary. I can successfully load web pages. However, I almost immediately start having trouble loading web pages. Looking at the firewall logs in OPNsense, I can see lots of traffic still being "allowed" out using the primary/WAN1 public IP address. However, tonight I did a packet capture on the secondary/WAN2, and found that all the packets had the appropriate secondary/WAN2 public IP address.

I'm not sure what to make of this, but it seems like there's a significant amount of traffic that's likely still trying to route via the primary/WAN1. I'm at a loss as to why that might be. Is there a common failover misconfiguration that might lead to something like this?

If you've read this far, thank you for your time and have a great day! :)

Version: OPNsense 20.7.7_1-amd64

Hello,

I've posted about this on Reddit a couple of times:

https://www.reddit.com/r/OPNsenseFirewall/comments/dcbyo8/meraki_concentrator_partially_blocked_digging/
https://www.reddit.com/r/OPNsenseFirewall/comments/d98aii/port_forward_not_working/

Basically, I have a strong suspicion that pf or similar software underpinning OPNsense is allowing only one network "flow" through each port, either forwarded or outbound. I have observed:

• Port forwards have appeared to work for Plex, but only for a single client at a time
• Port forwards appeared to work briefly with one of my BitTorrent trackers, it now reports I am "unconnectable"
• Meraki UDP hole punching used for AutoVPN appears to only be working for one of two sites now that it's behind OPNsense

That last observation is what really pointed me at this being a restriction on the number of allowed flows. It uses the same technology at every site, and OPNsense is actually the first firewall I've found that default blocks meshing (Meraki's source paper cited a 90%+ success rate for their technique, BTW).

The port forwarding I consider a serious issue because it doesn't behave as one would expect for a port-forward to behave. However, the behavior of normally restricting users to a single flow per outbound request, while stricter than most firewalls and likely to cause problems with marginal cases (I have a sneaking suspicion this is causing Skype problems for me), is an entirely reasonable choice.

With all that said, I don't really know how to troubleshoot this, much less change it. OpenBSD's packet filter documentation isn't awful, but it is tough to wade through for someone not versed in pf terminology.

Any help in figuring out how to move forward is much appreciated. Have a great day, all!