OPNsense Forum

English Forums => Hardware and Performance => Topic started by: cbb09 on October 20, 2019, 11:12:44 pm

Title: Weird speed issues with Mellanox ConnectX-3
Post by: cbb09 on October 20, 2019, 11:12:44 pm

Hi,

I installed a Mellanox ConnectX-3 10G nic today. The card was recognized after I prompted the system to load the module (I added mlx4en_load="YES" to /boot/loader.conf.local. So far so good.

However, when I ran iperf3 I noticed that I am only getting about 1.4GB/s on the interface. I've tried turning on/off all the hardware offloading settings, but to no avail. Speed doesn't exceed 1.4GB/s. To rule out other issues, I've booted a live CD image of Ubuntu 18.04.3 on the same system and ran iperf3 from there. I was getting about 9.85GB/s - which I expected.

Is there anything that I overlooked in OPNsense? Is it a module issue in FreeBSD?

Thanks!

Title: Re: Weird speed issues with Mellanox ConnectX-3
Post by: mimugmail on October 21, 2019, 05:24:52 am

Did you use the local iperf Plugin? These values arent really stable.
Also note GB is ok, in your case it's Gb, correct?

Title: Re: Weird speed issues with Mellanox ConnectX-3
Post by: cbb09 on October 22, 2019, 04:42:55 am

Quote from: mimugmail on October 21, 2019, 05:24:52 am

Did you use the local iperf Plugin? These values arent really stable.
Also note GB is ok, in your case it's Gb, correct?

I used iperf in the command line. Indeed, it’s Gb, not GB - sorry about that! I’ve also timed a large file transfer between my OPNsense box and another server on my LAN to rule out iperf issues. However, same result... I wonder if there’s a driver/module issue.

Title: Re: Weird speed issues with Mellanox ConnectX-3
Post by: mimugmail on October 22, 2019, 05:31:25 am

My last tests were with 19.1 and it was good. Maybe you build a Lab and test iperf in devices behind and infront