Messages

Opened github issue.

https://github.com/opnsense/core/issues/8574

[twice]

Using network IP addresses is inconvenient in my case due to the potentially large and unpredictable number of networks routed via OSPF. Maintaining aliases under such conditions would be impractical, tedious, or outright unfeasible.

But, after further investigation, I believe I have gained some new insight.

Packet flow in OPNsense, and possibly FreeBSD, appears to differ from RouterOS and potentially Linux. From what I understand (please correct me if I'm mistaken), a packet is inspected not once, but twice: once when it enters the firewall and again when it exits, provided it has been accepted in the first stage. When a packet enters the firewall, no routing decision has been made yet (or it's not exposed in pf/OPNsense), meaning the rules system does not know where the packet would be directed if accepted.

The behavior I aim to achieve could be implemented in two steps:

• Using an inbound rule for interface group A: when a packet enters interface A, we accept it and mark it with the tag from-interface-A.
• Using an outbound rule for interface group A: when a packet is about to exit toward interface A, and it carries the tag from-interface-A we accept it.

In this way, all packets originating from interface A are guaranteed to be allowed to exit toward interface A.

[I think I've figured it out.]

I think I've figured it out.

The routes for the remote networks were "inactive" because they referenced the remote gateway IP address alongside its corresponding remote network IP address.

In other words, in my setup, there isn't a common transit network between the hub and the three less advanced remote gateways, simply because they don't support multiple networks or IP addresses.

As a result, the remote gateway was formally unreachable since the remote site network wasn't covered by any other route. While the remote network and gateway were technically reachable via the auto-created routes by WireGuard, it seems these routes couldn't be taken into account.

I'm not certain if there's a concept akin to route scope in OPNsense/FreeBSD, but it appears that WireGuard routes cannot be used for next-hop searching in static routes configured via the System or Routing menu.

There are two potential solutions:

• Assign an additional IP address from the remote network to the WireGuard interface. This way, the remote network becomes connected, reachable, and advertised over OSPF without the need to propagate static routes.
• Configure static routes for the remote networks directly in the Routing/Static menu, without specifying the IP address of the remote gateway at the far end of the WireGuard tunnel—using only the WireGuard interface instead. Then, advertise static routes over OSPF as well.

Both approaches require maintaining the remote network IP addresses in two places.

A sample working configuration follows.

Code Select Expand

!
frr version 8.5.7
frr defaults traditional
hostname gw.lan.mydomain.tld
log syslog notifications
!
ip route 10.a.b.0/24 wg0
ip route 10.a.c.0/24 wg0
ip route 10.d.e.0/24 wg0
!
interface hn0
 ip ospf passive
exit
!
interface hn1
 ip ospf passive
exit
!
interface wg1
 ip ospf area 10.2x.0.0
 ip ospf network point-to-point
exit
!
interface wg2
 ip ospf area 10.x.0.0
 ip ospf network point-to-point
exit
!
interface ztdmpyyyyyyyyyyyyyy
 ip ospf area 10.x.0.0
 ip ospf network broadcast
exit
!
router ospf
 ospf router-id 10.x.0.1
 redistribute connected
 redistribute static
exit
!
end


[With "Disable routes" unchecked]

Hello everyone,

I've been experimenting with OPNsense for a few days to create a hub that securely connects to remote sites using both WireGuard and ZeroTier, while distributing routes via OSPF.

In addition to sites equipped with advanced routing systems, I also have three sites with basic routers that support WireGuard tunnels. To address this, I set up a WireGuard instance in the hub with three peers for these sites. After verifying connectivity through the tunnels, my plan was to distribute the routes for these sites via OSPF.

Unfortunately, despite trying several options for enabling static and/or kernel routes, these routes are not being advertised. Specifically:

1) With "Disable routes" unchecked in the WireGuard instance:

• Routes for the site networks are visible using netstat -rn.
• Routing from OPNsense to the sites works.
• However, vtysh doesn't display any kernel/static route apart from some Cloudflare DNS server entries (likely derived from the WAN interface).

Code Select Expand

Destination        Gateway            Flags         Netif Expire
0.0.0.0/1          link#8             US              wg0
default            10.a.b.1           UGS             hn0
1.0.0.2            10.a.b.1           UGHS            hn0
1.1.1.2            10.c.d.1           UGHS            hn0
10.e.f.1           link#8             UHS             wg0
10.g.h.0/24        link#8             US              wg0
10.i.j.0/24        link#8             US              wg0


Codes: K - kernel route, C - connected, S - static, R - RIP,
       O - OSPF, I - IS-IS, B - BGP, E - EIGRP, T - Table,
       v - VNC, V - VNC-Direct, A - Babel, f - OpenFabric,
       > - selected route, * - FIB route, q - queued, r - rejected, b - backup
       t - trapped, o - offload failure

K>* 0.0.0.0/0 [0/0] via 10.a.b.1, hn0, 06:53:28
K>* 1.0.0.2/32 [0/0] via 10.a.b.1, hn0, 06:53:28
K>* 1.1.1.2/32 [0/0] via 10.a.b.1, hn0, 06:53:28

wg0: flags=10080c1<UP,RUNNING,NOARP,MULTICAST,LOWER_UP> metric 0 mtu 1420
        description: bk_wg (opt2)
        options=80000<LINKSTATE>
        inet 10.k.l.129 netmask 0xfffffff0
        groups: wg wireguard bk
        nd6 options=1<PERFORMNUD>


2) With a gateway set up in System/Gateways and a static route in System/Routes:

• A kernel route appears in vtysh but is marked "inactive."
• There's no change in netstat -rn.

Code Select Expand

K   10.g.h.0/24 [0/0] via 10.g.h.1 inactive, 00:00:49

3) With a static route added in Routing/OSPF/Static:

• A static route appears in vtysh but is still marked "inactive."

Code Select Expand

S   10.g.h.0/24 [1/0] via 10.g.h.1, wg0 inactive, weight 1, 00:00:37


In all cases, routes fail to appear in OSPF. I've tried multiple configurations in OSPF without filters (filters could be a mystery for another night). I suspect this behavior is related to these routes being considered "inactive," preventing them from being advertised by OSPF.

Here is the current running configuration for FRR:

Code Select Expand

!
frr version 8.5.7
frr defaults traditional
hostname gw.lan....
log syslog notifications
!
ip route 10.g.h.0/24 10.g.h.1 wg0
!
interface hn0
 ip ospf passive
exit
!
interface hn1
 ip ospf passive
exit
!
interface wg1   #### P2P for site with OSPF router
 ip ospf area 10.x.0.0
 ip ospf network point-to-point
exit
!
interface wg2   #### P2P for site with OSPF router
 ip ospf area 10.x.0.0
 ip ospf network point-to-point
exit
!
interface ztdmp.....   #### ZeroTier L2 Domain for sites with OSPF router
 ip ospf area 10.x.0.0
 ip ospf network broadcast
exit
!
router ospf
 ospf router-id 10.x.0.1
 redistribute kernel
 redistribute connected
 redistribute static
exit
!
bfd
exit
!
end
Thanks for any input!

Hello everyone,

I've been experimenting with OPNsense for a few days now. I've also gone through the documentation and explored search results, but I haven't been able to find an answer to my question.

I would like to allow all traffic originating from one group of interfaces and directed to another group of interfaces without any additional filtering.

In RouterOS, this can be done in the Filter chain since the routing decision has already been made. However, I don't fully understand when OPNsense performs packet filtering, whether this is achievable, and if so, how to set it up.

I couldn't find any options in the UI to accomplish this.

Apparently the firewall rules defined as floating and in interface groups including the zerotier interface are not effective for the zerotier interface after reboot.

So for example after reboot incoming multicast traffic for OSPF is dropped despite a rule in an interface group.

And web gui access from zerotier is not allowed despite a floating rule.

On the other hand, a rule for ospf multicast added on the zerotier interface is applied since reboot.

Is this some kind of limit/bug or should I change some setting?
Any workaround? Replicating and maintaining all rules from group also in ZT interface is not easy...

Hello everyone,

I've been experimenting for the past few days, for the first time, with OPNsense 25.1.5_5 (amd64). I'm attempting to set up OSPF between an OPNsense machine and two Mikrotik routers.

I have established VPN connections using both WireGuard and ZeroTier:

• There's a ZeroTier L2 domain, shared by the three devices, configured in OSPF as multicast.
• Additionally, there are three point-to-point WireGuard connections, configured in OSPF as point-to-point.

Everything works as expected until I reboot OPNsense. After the reboot, only the WireGuard adjacencies are fully re-established. The Mikrotik routers show the OPNsense neighbor stuck in the Init state.

Here's what happens:

• Mikrotik routers send "Hello" packets via multicast.
• OPNsense also sends "Hello" packets via multicast.
• Mikrotik routers receive the "Hello" packets from OPNsense, which include its neighbor in the Init state, and this cycle continues indefinitely.

The only workaround I've found is to make a minor and seemingly irrelevant change to the OPNsense firewall rules and apply it (e.g., enabling or disabling an unrelated rule). This action re-establishes adjacency over ZeroTier.

Running pfctl -d; pfctl -e also temporarily solves the problem, but only for half a minute.

Does anyone have ideas or suggestions to address this issue?