Messages

Unfortunately it seems that it's not related to the costs but seems to be a bug at FRR.
I logged at Github ticket to investigate this behaviour: https://github.com/opnsense/plugins/issues/3846

From my point of view the issue is related to network type point to multipoint as there my error occurs.
Yesterday I changed to regular broadcast and this works without my issues.


In general costs should be fine as my direct connection from CCR2004 to CCR2116 should be not used at all as costs on both interfaces are set to 100 each while OPNsense1 is using 10 and OPNsense2 20.
All routers are in the same area and should be aware of all costs.

I think I fixed it by setting different costs for interface pointing to CCR2004 and CCR2116

Hello everyone,

I stumbled accross a weird routing behaviour on my network.
In general my network is:

Mikrotik CCR2004 as internet & VPN router connected to 2x OPNsense which are connected to a Mikrotik CCR2116 as my network router.
As a failover my CCR2004 is also direct to CCR2116 but with higher costs so any traffic would go through my firewall.

Anytime I modify an OSPF setting on OPNsense and reload the process it gets reconnected but no traffic is going through it - it becomes unreachable.

Both Mikrotik routers show that OSPF is connected and exchanged all information (State = Full).

I connected to an OPNsense VM to see what is happending there and FRRs vtysh also show that it's fully exchanged on I can see all routes.
Only ICMP and traceroute is not working:

Code Select Expand

PING k8s-1.hks.lan (10.0.22.80): 56 data bytes
92 bytes from 172.16.1.2: Time to live exceeded
Vr HL TOS  Len   ID Flg  off TTL Pro  cks      Src      Dst
4  5  00 0054 b660   0 0000  01  01 35e7 172.16.1.2  10.0.22.80

92 bytes from 172.16.1.2: Time to live exceeded
Vr HL TOS  Len   ID Flg  off TTL Pro  cks      Src      Dst
4  5  00 0054 ef8c   0 0000  01  01 fcba 172.16.1.2  10.0.22.80

92 bytes from 172.16.1.2: Time to live exceeded
Vr HL TOS  Len   ID Flg  off TTL Pro  cks      Src      Dst
4  5  00 0054 cbe3   0 0000  01  01 2064 172.16.1.2  10.0.22.80

Traceroute is running between Mikrotik and CCR2116 - I guess until TTL is reached and is then been kicked.


Currently the only solution is to restart CCR2116 to get everything running again.


Is there a bug somewhere?

Okay I think the main issue was that OSPF was stuck at the INIT state.
Not sure why OPNsense 1 did not sync with the other OPSF routers.

And as I initially wanted to set up a full working Nginx on just OPNsense 1 before I sync the settings over to OPNsense 2 I had those issues.

I think I found the issue:

My CARP VIP sits on OPNsense 1 while OPNsense 2 is acting a backup.

What is strange is that Nginx is using the VIP on my OPNsense 2 instead of 1st OPNsense.

I have configured my VIP including my /28 subnet.
And if I configure Nginx to exclusively use it I can also see that's listening on it. But it's not serving anything

Hello there,

I just started to test the Nginx plugin to use it as a reverse proxy.
When running it on the firewall IP it works as expected but as soon as I use my CARP IP I am no longer able to connect to the site.

Is this a know behaviour or a bug?

Hello there,

just a quick question:
Nginx WAF section offers three options:
- block XSS score
- block SQL injection score
- custom security policy

Would block XSS and SQL injection score not being needed if I apply the custom security policy with the available values?

Okay it seems to block the initial traffic but traffic from same IP with same attack vector (e.g. SSH scanning) will be passed through after some time

Hello everyone,

I am using Suricata for quiet a while on my virtual OPNsense firewall.
I recently stumbled accross an intressting thing: On Suricatas log it says that it blocks some specific IPs for e.g. SSH scan but on the destination host I can also see that fail2ban is banning the specific IP.
So from my point of view it looks like that Suricata is "lying" about blocking it.

Anyone else having same troubles?

Not sure if this is relevant for this topic but with my new router setup I am also using Suricata as IDS/IPS (from SELKS https://github.com/StamusNetworks/SELKS) with equal settings as on OPNsense.

With this setup it is not having any troubles with my LAN & remote sites.

According to the log files it blocked the traffic.
But also as mentioned earlier: Even if Suricata mentioned it blocked it for me it worked without any issues.

Thank you very much for your investigation.
What I forgot to mention is that I also created an "allow rule" for all my local and remote subnets.
Surricata still denied traffic.

Is this also an expected behaviour?

My setup was:

Remote site <-- Wireguard tunnel with OSPF --> Internet router (OSPF enabled) <-- 2x OPNsense with HA (OSPF enabled)

On both sides I had some AD domain controllers and windows server which used Microsoft typical ports (mostly RCP, SMB).

Surricata complained about both direction: From local site to remote but also from remote to local.


The subnet between OPNsense and my internet router was the WAN side according OPNsense firewall logic.