EXTREMELY Slow iperf3 through OPNSense LAN Interface & Static Routes

Started by luckylinux, June 14, 2024, 05:52:05 PM

Previous topic - Next topic
I am revisiting an issue I had a few Months ago, this time with IPv6, as opposed to the previous Issue I had with IPv4.

Desktop Main IP: 2XXX:XXXX:XXXX:0001:0000:0000:0003:0066/64

Server Main IP: 2XXX:XXXX:XXXX:0001:0000:0000:0008:0015/64

Containers Dedicated Subnet: 2XXX:XXXX:XXXX:ff15:0000:0000:0000:0000/64

OPNSense has:

  • Firewall -> Settings -> Advanced -> [CHECKED]  Bypass firewall rules for traffic on the same interface
  • Gateway: 2XXX:XXXX:XXXX:0001:0000:0000:0008:0015/64
  • Static Routes: 2XXX:XXXX:XXXX:ff15:0000:0000:0000:0000/64 via 2XXX:XXXX:XXXX:0001:0000:0000:0008:0015/64

If I setup the same Static Route on my Desktop, then I can iperf3 -c 2XXX:XXXX:XXXX:ff15:0000:0000:0000:0002 -P 10 and I get something like 500 Mbit/s. Fair enough for a gigabit connection with all Switches etc.

If I go THROUGH OPNSense I get 0.00 bytes/s ... 0.00 bytes/s  ???.

Is there maybe an issue "on the way back" ?

Like: Desktop 2XXX:XXXX:XXXX:0001:0000:0000:0003:0066 -> OPNSense 2XXX:XXXX:XXXX:0001:etc IN -> OPNSense 2XXX:XXXX:XXXX:ff15:etc OUT -> 2XXX:XXXX:XXXX:ff15::0002

But maybe there is no "route back" to the Desktop ? Or if the Connection is successfully established, then the reply Packets would go down that open connection Anyways ?

Nothing "red" in the Firewall Logs.

Hardware NIC Offloading was DISABLED ([CHECKED]), but ENABLED ([UNCHECKED]) did NOT make any difference.

EDIT 1: Iperf3 can go down to full speed (~ 900 Mbit/s) if, DURING THE RUN, I change something in OPNSense -> Firewall -> Advanced -> Settings -> click apply. But starting a new instance of iperf3 -c brings back again the speed to 0.00 bytes/s.

So of Course this is NOT a solution. Neither a workaround.

UDP iperf3 speed is around 1Mbit/s per each parallel thread (-P 10 -> 10 Mbit/s, -P 100 100 Mbit/s) whereas the default TCP iperf3 is 0.00 bytes/s UNLESS I change something in OPNSense -> Firewall -> Advanced -> Settings -> click apply DURING the iperf3 run.