DEC2750 and 10Gb Network

[normanos]

I bought DEC2750. Hope it 'sspec don't need.


Have 10G SFP+ SR Multimode Fibre Module- 10GBase-SR LC Transceivers (10Gtek).

I usi fibre cable to Mikrotik switch and fibtre cable to Gentoo Linux.

When I testing LANwith iperf3 from opnsense to gentoo, I can get max 3.32 Gbits/sec

I know, I will never need 10 Gbits, but why I can't get in LAN full speed? I done something wrong with setup or maybe its transceiver not arelly supported? In Linux systems they working fine.

Sorry about messy question, if need other details, just let me know.

Thank You!

[meyergru]

You will not get much more than this using only one thread. Try "iperf -P 4".

[normanos]

tried -P 4, -P 8 ... nothing. Even worst.

[ ID] Interval           Transfer     Bitrate         Retr
[  5]   0.00-10.01  sec   944 MBytes   791 Mbits/sec    0             sender
[  5]   0.00-10.01  sec   944 MBytes   791 Mbits/sec                  receiver
[  7]   0.00-10.01  sec   945 MBytes   791 Mbits/sec    0             sender
[  7]   0.00-10.01  sec   945 MBytes   791 Mbits/sec                  receiver
[  9]   0.00-10.01  sec   944 MBytes   791 Mbits/sec    0             sender
[  9]   0.00-10.01  sec   944 MBytes   791 Mbits/sec                  receiver
[ 11]   0.00-10.01  sec   944 MBytes   791 Mbits/sec    0             sender
[ 11]   0.00-10.01  sec   944 MBytes   791 Mbits/sec                  receiver
[SUM]   0.00-10.01  sec  3.69 GBytes  3.17 Gbits/sec    0             sender
[SUM]   0.00-10.01  sec  3.69 GBytes  3.17 Gbits/sec                  receiver

When I trying from one Linux to other Linux, I getting all 10gb/s and without that -P 4

[ ID] Interval           Transfer     Bandwidth       Retr  Cwnd
[  4]   0.00-1.00   sec  1.09 GBytes  9.36 Gbits/sec    3    588 KBytes       
[  4]   1.00-2.00   sec  1.09 GBytes  9.33 Gbits/sec    0    632 KBytes       
[  4]   2.00-3.00   sec  1.09 GBytes  9.40 Gbits/sec    0    632 KBytes       
[  4]   3.00-4.00   sec  1.09 GBytes  9.41 Gbits/sec    0    740 KBytes       
[  4]   4.00-5.00   sec  1.09 GBytes  9.35 Gbits/sec    0    740 KBytes       
[  4]   5.00-6.00   sec  1.09 GBytes  9.39 Gbits/sec    0    740 KBytes       
[  4]   6.00-7.00   sec  1.09 GBytes  9.36 Gbits/sec    0    740 KBytes       
[  4]   7.00-8.00   sec  1.09 GBytes  9.36 Gbits/sec    0    740 KBytes       
[  4]   8.00-9.00   sec  1.09 GBytes  9.40 Gbits/sec    0    740 KBytes       
[  4]   9.00-10.00  sec  1.09 GBytes  9.39 Gbits/sec    0    740 KBytes       
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bandwidth       Retr
[  4]   0.00-10.00  sec  10.9 GBytes  9.37 Gbits/sec    3             sender
[  4]   0.00-10.00  sec  10.9 GBytes  9.37 Gbits/sec                  receiver

i seen in other post that need to disable suricata, it never was enabled.  Don't have special configuration, just using for 2 WAN's. Have ports forwarded and OpenVPN what not in use.

[lilsense]

what makes you think it's not mikrotik or the gentoo linux?

[normanos]

Because Mikrotik acts as switch only.  I have several Linux with 10Gb fibre cards, from any of them works fine.
  That's why I wondering why it's not working with 10Gb/s.

Linux - Mikrotik - Linux
Opnsense - Mikrotik - Linux.

[lilsense]

turn off your ips and zenarmor and retest...

[normanos]

@lilsense, You not reading what I writing? I told that have basic firewall with 2 WAN's.

Please don't offer thing wich already was offered in other posts.  I went thru them before making new topic.

I don't have any Virtual Ip's or zenarmor.

[rungekutta]

If you look back through previous threads you'll find examples both of people with good solid 10Gb performance and others who are struggling to get near line rate despite strong hardware. Fwiw, iperf against opnsense in itself seems to often disappoint but that may actually not be indicative of forwarding/firewall performance *through* the firewall. Have you tested this as well?

[normanos]

I'm trying from Opnsense LAN interface to LAN, because at the moment only LAN interface have 10 Gb/s network.

Yes, I also think it's simply iperf3 who failing, maybe it's some another way to check LAN speed?

[rungekutta]

I guess it depends on what your end goal is. At the end of the day you'll want to measure the effectiveness of opnsense in the way that you intend to use it. Iperf could be directly relevant if you intended to host data-heavy services on opnsense itself. Which I'm presuming you're not. Granted a bit concerning that some bottleneck prevents it from pushing full line speed - but may actually turn out to be completely irrelevant in your real-world scenarios.

[meyergru]

Just a quick hint: hardware CRC offloading does not work on the ax interfaces (see https://forum.opnsense.org/index.php?topic=23339.msg141202#msg141202).

[Vilhonator]

Crazy advice but test realtime scenario.  Connect 2 or more computers and send / download 100+GB worth of data (10Gb/s / 8 = 1.250GB give or take), or bunch of computers doing that :D

I never trust iperf3 since it isn't 100% accurate.