Messages

Found the issue: "Hardware checksum offload" was not deactivated.

Strange thing, the setting was not even present in the configuration. I'm guessing that this setting appear in a recent update and was not set to default, which is activated.

Anyway, my question remains: how would you trace a packet though the OPNSense box to debug this kind of behavior?

Hello Everyone,
For the last few days, I'm seeing very slow traffic when going though the firewall, an OPNSense (v.20.1.8 ) hosted on OVH's Public Cloud and filtering/routing traffic between private networks. I didn't changed anything on OPNSense configuration.

CPU is idle most of the time and there is plenty of free RAM.

• Load average : 0.27, 0.20, 0.17
• Memory usage : 12 % ( 960/7963 MB )

I read a lot of post regarding performance, so I can already tell you that I disabled proxy and IPS.
Also ran iperf3 test across the network :
From Client (OpenVPN client) to Server :

Code Select Expand

Accepted connection from x.x.0.2, port 56960
[  5] local x.x.12.51 port 5201 connected to x.x.0.2 port 56962
[ ID] Interval           Transfer     Bandwidth
[  5]   0.00-1.00   sec  9.83 MBytes  82.4 Mbits/sec
[  5]   1.00-2.00   sec  10.7 MBytes  89.5 Mbits/sec
[  5]   2.00-3.00   sec  10.6 MBytes  89.0 Mbits/sec
[  5]   3.00-4.00   sec  10.3 MBytes  86.2 Mbits/sec
[  5]   4.00-5.00   sec  11.1 MBytes  92.7 Mbits/sec
[  5]   5.00-6.00   sec  10.8 MBytes  90.9 Mbits/sec
[  5]   6.00-7.00   sec  8.11 MBytes  68.0 Mbits/sec
[  5]   7.00-8.00   sec  10.9 MBytes  91.8 Mbits/sec
[  5]   8.00-9.00   sec  10.3 MBytes  86.3 Mbits/sec
[  5]   9.00-10.00  sec  10.6 MBytes  88.7 Mbits/sec
[  5]  10.00-10.04  sec   445 KBytes  84.0 Mbits/sec
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bandwidth
[  5]   0.00-10.04  sec   104 MBytes  86.6 Mbits/sec                  sender
[  5]   0.00-10.04  sec   104 MBytes  86.6 Mbits/sec                  receiver

All is fine.
But from Server to Client (-R option on iperf) :

Code Select Expand

Accepted connection from x.x.0.2, port 57062
[  5] local x.x.12.51 port 5201 connected to x.x.0.2 port 57063
[ ID] Interval           Transfer     Bandwidth       Retr  Cwnd
[  5]   0.00-1.00   sec  34.4 KBytes   282 Kbits/sec   10   2.65 KBytes
[  5]   1.00-2.00   sec  0.00 Bytes  0.00 bits/sec    1   2.65 KBytes
[  5]   2.00-3.00   sec  0.00 Bytes  0.00 bits/sec    0   2.65 KBytes
[  5]   3.00-4.00   sec  0.00 Bytes  0.00 bits/sec    1   2.65 KBytes
[  5]   4.00-5.00   sec  0.00 Bytes  0.00 bits/sec    0   2.65 KBytes
[  5]   5.00-6.00   sec  0.00 Bytes  0.00 bits/sec    0   2.65 KBytes
[  5]   6.00-7.00   sec  26.5 KBytes   217 Kbits/sec   11   2.65 KBytes
[  5]   7.00-8.00   sec  79.4 KBytes   651 Kbits/sec   17   2.65 KBytes
[  5]   8.00-9.00   sec  71.5 KBytes   585 Kbits/sec   16   2.65 KBytes
[  5]   9.00-10.00  sec  82.0 KBytes   672 Kbits/sec   14   2.65 KBytes
[  5]  10.00-10.05  sec  0.00 Bytes  0.00 bits/sec    0   3.97 KBytes
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bandwidth       Retr
[  5]   0.00-10.05  sec   294 KBytes   240 Kbits/sec   70             sender
[  5]   0.00-10.05  sec   262 KBytes   214 Kbits/sec                  receiver


I also tried between two hosts on same vlan : 100Mbit/s in both direction.

And between two hosts in different VLAN (routing through OPNSense), traffic is slow (few Mbps instead of 100Mbps), but better than through VPN :

Code Select Expand

Accepted connection from x.x.11.51, port 37098
[  5] local x.x.12.51 port 5201 connected to x.x.11.51 port 37100
[ ID] Interval           Transfer     Bandwidth
[  5]   0.00-1.00   sec   187 KBytes  1.53 Mbits/sec
[  5]   1.00-2.00   sec   543 KBytes  4.45 Mbits/sec
[  5]   2.00-3.00   sec   608 KBytes  4.98 Mbits/sec
[  5]   3.00-4.00   sec   655 KBytes  5.36 Mbits/sec
[  5]   4.00-5.00   sec   450 KBytes  3.68 Mbits/sec
[  5]   5.00-6.00   sec   793 KBytes  6.50 Mbits/sec
[  5]   6.00-7.00   sec   768 KBytes  6.29 Mbits/sec
[  5]   7.00-8.00   sec   601 KBytes  4.92 Mbits/sec
[  5]   8.00-9.00   sec   492 KBytes  4.03 Mbits/sec
[  5]   9.00-10.00  sec   638 KBytes  5.22 Mbits/sec
[  5]  10.00-10.04  sec  22.6 KBytes  4.65 Mbits/sec
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bandwidth       Retr
[  5]   0.00-10.04  sec  5.75 MBytes  4.81 Mbits/sec  749             sender
[  5]   0.00-10.04  sec  5.62 MBytes  4.70 Mbits/sec                  receiver


Using top, I can see :

Code Select Expand

CPU:  0.0% user,  0.0% nice,  0.2% system,  1.3% interrupt, 98.5% idle
Interrupt is going to 2-3% at max. Is it a problem?

My feeling is that packet processing take more time than it should, which reduce the bandwidth. How can I debug that and guess why it changed like that.

Thank you for your help,
K4ngoo