Gateway group being ignored [SOLVED]

[lordxenu]

Hi folks,

I'm scratching my head a lot so i'm reaching out for a little bit of help please. I've been running OPNSense for many years and while i'm nowhere near an expert at it, i'm pretty comfortable. I have a reasonably complex setup with multiple VLANs , multiple gateways etc.

Anyhow, I have a gateway group defined that i call "Secure failover". This means that this group sends all my traffic to my OpenMPTCPRouter setup, but if that gateway is down, then it sends all traffic via my wireguard to my VPS.

In the last 2 days, suddenly it ignores my firewall configuration for ALL my local networks and sends all data to my DSL connection instead of this gateway group. The gateway group is monitored, it's green, it's alive, but OPNSense is ignoring the rule that sends traffic to this group and is just sending it straight out via the DSL connection. If I turn off the DSL connection, it sends it out via my backup 5G connection.

Neither of these should be used by my LAN computers. They should as mentioned use either my OMR router as the gateway, or failing that, the Wireguard connection. But it's just ignoring it.

In my LAN network, the rule I have says

   IPv4 *   *   *   *   *   SECURE_FAILOVER   *

So that should route all traffic via that gateway, but it's just not doing it. This rule has been in place for months and has worked fine. Until recently. And, the only thing that has changed is that I had to turn off and back on my DSL connection. Which really shouldn't have made a difference.

As far as I can tell, there are no other rules that are overriding this, and nothing to tell my LAN to use the DSL connection as the gateway. OPNSense thinks the secure_failover gateway group is up and running, but it's just not using it.

I really don't know why. Does anyone have any suggestions please?

[lordxenu]

Here's another oddity, from my laptop, on the LAN network

(10.0.0.1 = OPNSense)
(2.XXXX   = DSL)
(192.168.100.1 = OpenMPTCPRouter)


mf@ASH:~$ mtr www.google.com -r
Start: 2024-07-22T17:41:13+0100
HOST: ASH                         Loss%   Snt   Last   Avg  Best  Wrst StDev
  1.|-- ASH.mshome.net             0.0%    10    0.2   0.3   0.2   0.8   0.2
  2.|-- 10.0.0.1                   0.0%    10    0.6   0.5   0.4   0.7   0.1
  3.|-- 2.x.x.x                0.0%    10    2.7   2.7   2.6   3.0   0.1
  4.|-- ip-89-200-132-9.ov.easyne  0.0%    10    5.3   5.5   5.1   6.0   0.3
  5.|-- 2.120.13.239               0.0%    10    5.2   5.3   4.6   8.0   1.0
  6.|-- 209.85.249.187             0.0%    10    7.7   7.1   5.8   8.7   0.9
  7.|-- 192.178.46.81              0.0%    10    6.0   6.4   6.0   7.0   0.3
  8.|-- prg03s02-in-f100.1e100.ne  0.0%    10    6.8   7.0   6.1  10.4   1.2

mf@ASH:~$ tracepath www.google.com
 1?: [LOCALHOST]                      pmtu 1500
 1:  ASH.mshome.net                                        0.208ms
 1:  ASH.mshome.net                                        0.116ms
 2:  10.0.0.1                                              0.359ms
 3:  192.168.100.1                                         0.786ms
 4:  10.255.252.1                                         42.411ms


but going to whatismyip.com / whatismyipaddress.com / ipv4.canhazip.com  all give the DSL public IP.

[lordxenu]

Eh sorry this looks like it's not a problem with OPNSense at all.
Anyway I restored from backups and i'm back up and running but I think this was some weird networking issue with OMR not OPNSense.

Please disregard.