Migrating OpenBSD pf to OpnSense 26.7 - routing issues

[mbrogies]

Hi Folks,

my first post here on the forum.

In my past, I have installed already several OpnSense, often as something completely for a new environment.

This time it's more complex and somehow I have a interesting problem with routing.

The old setup:

Code Select Expand

#                    +----------+
#                    |  router  | ISP Router (Fiber)
#                    +----+-----+
#                          |
#                          |
#                    +----+-----+
#                    |          |
#                    |          |
#  +-----------------+  Hub     +------------------+                   
#  |WANIP            +----+-----+                  |  WANIP VPN
# +-+----------+          |                +-------+------+       
# | fortinet  |           |bge1            |  OpenBSD/IKED|
# +-+----------+          |WAN IP         +-------+------+     
#  |                 +-----+------+                | 10.2.1.254
#  +-----------------|  firewall  +-----------------+         
#  em0/192.168.200.1 |            | bge3/10.2.1.1             
#                    |            |                         
#                    |            |                         
#  DMZ  -------------+            +---------------------------+10.4.1.254 - WLAN
#  em1/172.32.16.1   |            | bge2/10.4.1.1           
#                    |            |                         
#  DMZ2 -------------+            |                         
#  em2/10.3.1.1      |            |                         
#                    |            |
#                    +-----+------+
#                          |bge0
#                          |192.168.10.254
#                          |
#                          |
#                          |
#                          |
#                          | 192.168.10.1
#                          | 192.168.100.1
#                          | 192.168.140.1
#                    +-----+----+
#                    | Layer3SW +---------------192.168.10.0/24
#                    +-----+----+
#                          |
#                         / \
# 192.168.140.0/24--------   -------------- 192.168.100.0/24                           

The firewall in this case is an OpenBSD with pf

Now I try to replace "firewall" with an OPNsense in the first step.
Afterwards there will be some additional improvents (VLAN / true segmentation etc.)
Currently I am just focusing replaceing the old stuff with a current HW/SW.


And here some intresting routing problem begins for me.

Server1:
IP 192.168.10.33/24
GW: 192.168.10.1

Server2:
IP: 172.32.16.20/24
GW: 172.32.16.1

Example packet flow: 192.168.10.33 -> 172.32.16.20

192.168.10.33 -> 192.168.10.1 -> 192.168.10.254 -> 172.32.16.1 -> 172.32.16.20
Both IPs (192.168.10.254,172.32.16.1) are Interfaces on the OPNsense.

This works like a charm. The Server 1 can access everything needed on 172.32.16.20 (Server 2).

The other way around is currently making me some headache.

172.32.16.20 -> 172.32.16.20 -> 192.168.10.254 -> 192.168.10.33

I am stepping here in some sort of asymetric routing, since the OPNSense Interface is on the same Network as the Server 1 it try's to send the traffic directly back to Server 1 instead over the Layer3SW. This somehow let the statetable not match the traffic and the packages are silently dropped.

In the Livelog GUI i just see the "pass" packets and nothing else, even after enabled logging of "everything". NAT, auto generated rules, default block, etc.


Do you guys have any clue how I can solve that?

With the old OpenBSD this is running since years.


Greetings,

Matthias