Messages

Okay this one is weird.

Unplugged every device on the switch and had packet loss from both servers to the OPNsense. I plugged my laptop with the usb nic into the switch and ran an iperf3 test.
No packet loss there. Same switch port configuration.

What could it be?😪

Yep that was it...
Found the device, a security camera...

Now theres some packet loss in the management network (vlan 2) to solve, I will look into that.
Thanks so far!

I have some news!
Did some tests with the usb network card and figured out that the packet loss happens when vlan 30 is mapped to the nic.

Since the USB nic quickly reached its limits (ping: sendto: No buffer space available), I removed it and was able to determine the same behavior with the built-in nic.
Sounds like a switch issue, I'll dig into tho and report back.

Thanks for help so far!

It doesn't only happen, when the OPNsense is the target.
The test happens in a local network from two devices which are directly connected to the switch, no routing happens.
If the OPNsense is unplugged from the switch, the packet loss immediately vanishes.
Disabled hardware acceleration and rebooted afterwards. Changed nothing.

I did replace the cables already. Didn't change anything.
I also don't understand in which aspect the OPNsense/NIC could negatively influence the traffic on the switch. I mean, the iperf devices were in the same vlan and same network. Thus routing doesn't happen.
I'm also not sure how much a different os could bring me forward, because I'd somehow have to replicate the network config.

I have a usb nic. Maybe I should try this one first?

Hi,

i did some iperf3 tests the last days and figured out, that packet loss appears in all networks, when the OPNSense MiniPC (i226-V) is connected to my Unifi Switch. The interesting this is, that iperf3 only shows packet loss in one direction.

The problem immediately vanishes, when I disconnect the OPNSense from the switch.
Connecting the switch to an unconfigured port on the OPNSense, doesn't cause the issues.

The two servers and notebooks I used for the test are in the same network & vlan.

Code Select Expand

root@TrueNAS[~]# iperf3 -c 192.168.1.100
Connecting to host 192.168.1.100, port 5201
[  5] local 192.168.1.46 port 52000 connected to 192.168.1.100 port 5201
[ ID] Interval           Transfer     Bitrate         Retr  Cwnd
[  5]   0.00-1.00   sec   113 MBytes   945 Mbits/sec  199    242 KBytes       
[  5]   1.00-2.00   sec   110 MBytes   922 Mbits/sec  151    195 KBytes       
[  5]   2.00-3.00   sec   110 MBytes   924 Mbits/sec  121    264 KBytes       
[  5]   3.00-4.00   sec   111 MBytes   930 Mbits/sec  157    236 KBytes       
[  5]   4.00-5.00   sec   111 MBytes   931 Mbits/sec  117    294 KBytes       
[  5]   5.00-6.00   sec   109 MBytes   914 Mbits/sec  145    267 KBytes       
[  5]   6.00-7.00   sec   111 MBytes   930 Mbits/sec  150    243 KBytes       
[  5]   7.00-8.00   sec   110 MBytes   923 Mbits/sec  118    276 KBytes       
[  5]   8.00-9.00   sec   111 MBytes   933 Mbits/sec  114    310 KBytes       
[  5]   9.00-10.00  sec   109 MBytes   918 Mbits/sec  160    270 KBytes       
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bitrate         Retr
[  5]   0.00-10.00  sec  1.08 GBytes   927 Mbits/sec  1432             sender
[  5]   0.00-10.00  sec  1.08 GBytes   925 Mbits/sec                  receiver

iperf Done.
root@TrueNAS[~]# iperf3 -c 192.168.1.100 -R
Connecting to host 192.168.1.100, port 5201
Reverse mode, remote host 192.168.1.100 is sending
[  5] local 192.168.1.46 port 56234 connected to 192.168.1.100 port 5201
[ ID] Interval           Transfer     Bitrate
[  5]   0.00-1.00   sec   111 MBytes   932 Mbits/sec                 
[  5]   1.00-2.00   sec   112 MBytes   940 Mbits/sec                 
[  5]   2.00-3.00   sec   112 MBytes   941 Mbits/sec                 
[  5]   3.00-4.00   sec   112 MBytes   941 Mbits/sec                 
[  5]   4.00-5.00   sec   112 MBytes   940 Mbits/sec                 
[  5]   5.00-6.00   sec   112 MBytes   941 Mbits/sec                 
[  5]   6.00-7.00   sec   112 MBytes   941 Mbits/sec                 
[  5]   7.00-8.00   sec   112 MBytes   941 Mbits/sec                 
[  5]   8.00-9.00   sec   112 MBytes   941 Mbits/sec                 
[  5]   9.00-10.00  sec   112 MBytes   939 Mbits/sec                 
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bitrate         Retr
[  5]   0.00-10.00  sec  1.10 GBytes   942 Mbits/sec    0             sender
[  5]   0.00-10.00  sec  1.09 GBytes   940 Mbits/sec                  receiver

iperf Done.

Code Select Expand

root@pangolin:~# ip -s link show vmbr0
5: vmbr0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP mode DEFAULT group default qlen 1000
    link/ether 0c:c4:7a:a9:21:8f brd ff:ff:ff:ff:ff:ff
    RX:    bytes  packets errors dropped  missed   mcast           
    188181120880 32354906      0  525486       0 2137344
    TX:    bytes  packets errors dropped carrier collsns           
    133855832187 16058567      0       0       0       0

Code Select Expand

6: br0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP mode DEFAULT group default qlen 1000
    link/ether 5a:b6:1d:01:b2:34 brd ff:ff:ff:ff:ff:ff
    RX:    bytes  packets errors dropped  missed   mcast           
    109392092671 25760900      0  540315       0 2409010
    TX:    bytes  packets errors dropped carrier collsns           
     98245425617 10384107      0       0       0       0


What could cause this?
Thanks in advance!


~May

It works now...

Hi,

i found out, that I cannot add multiple IP aliases for for the same ipv6-link-local address on different interfaces.
Message: "Address already assigned."

I think that something that should get implemented.


~May

Wireguard with IPv6 configured also breaks RA.

root@matrix-synapse:~# ip -6 neigh
2a02:XXXX:1d:5200::f:3 dev eth0 FAILED
fe80::7e2b:e1ff:fe13:982e dev eth0 lladdr 7c:2b:e1:13:98:2e router STALE
2a02:XXXX:1d:5200:7e2b:e1ff:fe13:982e dev eth0 lladdr 7c:2b:e1:13:98:2e router STALE
2a02:XXXX:1d:5200::f:4 dev eth0 FAILED
2a02:XXXX:1d:5200::e dev eth0 FAILED

2a02:XXXX:1d:5200::e -> Virtual IP with deny service binding configured
2a02:XXXX:1d:5200::f:4 -> Wireguard Endpoint IP
2a02:XXXX:1d:5200::f:3 -> Wireguard Endpoint IP

I enabled the option. Now the RAs are correct, but Internet stiill does not work :/

If I remove the VirtualIP, it directly works again.

Hi Franco,

sorry for being uncertain.

I set the following address in virtualIP: 2a02:FFFF:1d:5200::e/64
I use it to reach the HAProxy on the OPNsense. It works.

However when I restart radvd after adding the VirtualIP, it gets announced to the clients in the 2a02:FFFF:1d:5200/64 subnet. That's the thing I'd like to prevent.

Heyy,

my virtual IP setting causes the in virtual IP configured IP to get advertised. That's a problem because the virtualIP I set isn't intended for that use.
In "Services: Router Advertisements" under the "Source Address" field, the only option is "Automatic". Not very helpful.

And help is appreciated!

Well Monit works great, except it didn't autostart too, and I found out why... I had a script in

Code Select Expand

/usr/local/etc/rc.syshook.d/start/ which paused a lot of stuff...


I added a configd action for the script & all services came up as expected.


Thanks for the help!

I can't even see in the logs that WireGuard trys to start. I'll look into Monit and report back.

It's cable internet, so there's a modem in front of the OPNsense box.
Hm I see. Is there a reason why there isn't a target like "wait until online" and then start services xyz.

Uhh, where could I find/enable that cronjob? I just see "Renew DNS for WireGuard on stale connections" here.