My LAN internal data transfer was not just slower than expected, but it was even slower than a download from the internet. I ran into this issue and couldn't figure out the cause, so i started writing things down to create a forum post. At the end i was able to fix it myself and decided to still post it here, because it might help someone else.
- Have i tried to turn it off and on again? Yes.
- There is no traffic shaping configured in OPNsense.
- The CPU load on the OPNsense box doesn't seem to be much affected by traffic.
- OPNsense box is APU board with CPU AMD GX-412TC SOC (4 cores).
- Desktop-PC 1 has a 1000 MBit NIC and is connected to LAN 1 of the OPNsense box.
- Desktop-PC 2 has a 1000 MBit NIC and is connected to LAN 2 of the OPNsense box.
Test: 100 MByte transfer from internet to PC 1
Protocol: HTTPS
Duration: 22 seconds
Transfer rate: 4.8 MByte/second
Note: This is a good transfer rate for WAN to LAN.
Test: 100 MByte transfer from internet to PC 2
Protocol: HTTPS
Duration: 21 seconds
Transfer rate: 4.8 MByte/second
Note: This is a good transfer rate for WAN to LAN.
Test: 500 MByte transfer from PC 2 to PC 1
Protocol: FTP
Duration: 2m 14s
Transfer rate: 3.7 MByte/second
Note: This is a bad transfer rate for LAN to LAN. Even with my slow harddrive i expected a much higher rate.
Test: 500 MByte transfer from PC 1 to PC 2
Protocol: HTTP
Duration: 7m 33s
Transfer rate: 1.4 MByte/second
Note: This is a very bad transfer rate for LAN to LAN. Even with my slow harddrive i expected a much higher rate.
Test: 500 MByte transfer from PC 1 to OPNsense box
Protocol: scp
Duration: 19 seconds
Transfer rate: 26 MByte/second
Note: This is a good transfer rate for LAN to LAN.
Test: 500 MByte transfer from OPNsense box to PC 1
Protocol: scp
Duration: 27 seconds
Transfer rate: 18 MByte/second
Note: This is a good transfer rate for LAN to LAN.
When i first noticed the issue, the OPNsense interface configuration "Speed and duplex" was set to "Default" for both LAN 1 and LAN 2. For testing i changed this to "100baseTX full-duplex", but the transfer rate didn't change. I reverted the changes back to "Default".
In the end i got this fixed by replacing the cable between PC 2 and LAN 2. I don't know if the old cable is broken or just not plugged in correctly, but i know that the new cable increased the transfer rate between LAN 1 and LAN 2 to at least 40 MByte/second :-)
Hi
Dodgy cable ? There is always some.... :)
If not using network switch with packet statistics, one may have a look onto several things to see if the low network layers are fine:
ifconfig
netstat -n -d -i
then going higher protocols, for example:
netstat -s -p tcp
If not 'ifconfig | grep media' , then dropped/error packets might diagnose that there is a bad cabling.
Must be checked on both sides of the cable.
T.
@testo_cz Thank you for your additional information!
I didn't even look for packet loss, because the transfer rates seemed to be stable. Or not. When i now think back, i remember there was at least one test with a "flapping" transfer rate.
Anyway, this is very helpful information, good to know for the next time debugging :-)