Suddenly high packet loss rate between OPNsense and Fritzbox

[cyberganny]

Hallo Community,

since 3 days I do have massive issues with the combination of my OPNSense Cluster and two attached Fritzboxes.

I see massive packet loss rates up to 50% on the connection of the different cluster nodes and the attached Fritzboxes. The Fritzboxen are direkt connected vie Lan cables, no switch involved. I already checked the cables, they are fine. Pings to OPNSense nodes from inside the LAN are getting top rates. When I log into the OPNSense nodes and start pings to the Fritzboxes I get these results:

Code: [Select]

root@fw-master:~ # ping 192.168.188.1
PING 192.168.188.1 (192.168.188.1): 56 data bytes
64 bytes from 192.168.188.1: icmp_seq=0 ttl=64 time=78.418 ms
64 bytes from 192.168.188.1: icmp_seq=1 ttl=64 time=4.887 ms
64 bytes from 192.168.188.1: icmp_seq=2 ttl=64 time=0.585 ms
64 bytes from 192.168.188.1: icmp_seq=3 ttl=64 time=30.765 ms
64 bytes from 192.168.188.1: icmp_seq=4 ttl=64 time=99.968 ms
64 bytes from 192.168.188.1: icmp_seq=5 ttl=64 time=285.366 ms
64 bytes from 192.168.188.1: icmp_seq=6 ttl=64 time=0.715 ms
64 bytes from 192.168.188.1: icmp_seq=8 ttl=64 time=0.585 ms
64 bytes from 192.168.188.1: icmp_seq=9 ttl=64 time=227.395 ms
64 bytes from 192.168.188.1: icmp_seq=10 ttl=64 time=0.746 ms
64 bytes from 192.168.188.1: icmp_seq=11 ttl=64 time=3.116 ms
64 bytes from 192.168.188.1: icmp_seq=12 ttl=64 time=0.603 ms
64 bytes from 192.168.188.1: icmp_seq=13 ttl=64 time=0.787 ms
64 bytes from 192.168.188.1: icmp_seq=14 ttl=64 time=0.543 ms
A complet inconsistent result. The RTTd values are therefor in 3 digits.

The adapter configs:

Code: [Select]

root@fw-master:~ # ifconfig em1
em1: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 1500
options=852098<VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM,WOL_MAGIC,VLAN_HWFILTER,VLAN_HWTSO>
ether 00:e0:67:09:5d:05
media: Ethernet autoselect (1000baseT <full-duplex>)
status: active
nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>

Code: [Select]

root@fw-master:~ # ifconfig lagg0
lagg0: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 1500
options=852098<VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM,WOL_MAGIC,VLAN_HWFILTER,VLAN_HWTSO>
ether 00:e0:67:09:5d:04
inet6 fe80::2e0:67ff:fe09:5d04%lagg0 prefixlen 64 scopeid 0x9
inet 10.x.x.101 netmask 0xffffff00 broadcast 10.x.x.255
inet 10.x.x.1 netmask 0xffffff00 broadcast 10.x.x.255 vhid 1
laggproto failover lagghash l2,l3,l4
laggport: em0 flags=5<MASTER,ACTIVE>
groups: lagg
carp: MASTER vhid 1 advbase 1 advskew 0
media: Ethernet autoselect
status: active
nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
Is it possible that the last updates of OPNSense delivered Ethernet driver updates or new adapter configs that results in these problems with Fritzboxes, maybe also with other devices?

Recently installed:

Code: [Select]

OPNsense 21.1.4-amd64
FreeBSD 12.1-RELEASE-p15-HBSD
OpenSSL 1.1.1k 25 Mar 2021
Am I the only one with that issue?
I am thankful for every helpful hint.

-Micha

[marcquark]

Did you try unpluggin the Fritzbox and check packet loss from another device/platform? Just to get an understanding whether there's a problem with your Fritzbox or with OPNsense.

Also, did you just (visually) check/inspect the cables or actually replace them? What are the RX/TX error rates on your interface? If they're above 0, that indicates a layer2 problem.

[cyberganny]

Problem is solved, it was an NAT issue with the multi WAN configuration.