Home
Help
Search
Login
Register
OPNsense Forum
»
Archive
»
20.7 Legacy Series
»
Mellanox ConnectX-3 support
« previous
next »
Print
Pages: [
1
]
Author
Topic: Mellanox ConnectX-3 support (Read 3871 times)
cyrus104
Newbie
Posts: 39
Karma: 1
Mellanox ConnectX-3 support
«
on:
November 21, 2020, 10:18:04 am »
I was able to follow mimugmail's instructions that were posted to the forum and to the site at the bottom of this post. I was able to get the driver to autoload on boot and updated the firmware.
These cards are able to do 56Gbps instead of just 40Gbps if being used with an FDR cable and capable switch such as the SX6036 I'm using. I have this card in some ESXi machines and everything is detected as 56Gbps as expected.
I would like to see if there is a way to set these cards at the faster speed in OPNsense, the interface settings only offer autoselect, 1G, 10G, and 40G right now.
I would also like to see if there are specific options to enable OPNsense to take "full" advantage of these cards such as hardware offloading. I am using VLANs so, the hardware offloading might not work.
Any help would be greatly appreciated.
https://www.routerperformance.net/opnsense/mellanox-connecx-management-in-opnsense/
Logged
mimugmail
Hero Member
Posts: 6766
Karma: 494
Re: Mellanox ConnectX-3 support
«
Reply #1 on:
November 21, 2020, 12:13:00 pm »
Please dont use offloading, it will break couple of features.
How much throughput so you achieve?
With Chelsio I got 26Gbit max
Logged
WWW:
www.routerperformance.net
Support plans:
https://www.max-it.de/en/it-services/opnsense/
Commercial Plugins (German):
https://opnsense.max-it.de/
cyrus104
Newbie
Posts: 39
Karma: 1
Re: Mellanox ConnectX-3 support
«
Reply #2 on:
November 22, 2020, 04:09:48 am »
I haven't gotten far enough along in rebuilding my network to get a full iperf using the 56gbps cards. I have 1 installed in my OPNsense box. I'm installing them in my 3 ESXi machines and a TrueNAS box today. That should give me a much better understanding of their performance.
Part of my issues was not knowing the link speed that the OPNsense actually negotiated to.
I wanted to see what you would put for all of the hardware offloading options. I uploaded a picture of my config, I think I've disabled all offloading. I left VLAN hardware filtering on but can turn that off if you recommend it.
The iperfs that I have done are from other machines with 10Gbps which all pull around 9.4Gbps if I run: iperf -c -p -P 20
Thanks for the quick response.
Logged
mimugmail
Hero Member
Posts: 6766
Karma: 494
Re: Mellanox ConnectX-3 support
«
Reply #3 on:
November 22, 2020, 08:52:40 am »
It shouldnt influence the Performance that much when disabled
Logged
WWW:
www.routerperformance.net
Support plans:
https://www.max-it.de/en/it-services/opnsense/
Commercial Plugins (German):
https://opnsense.max-it.de/
cyrus104
Newbie
Posts: 39
Karma: 1
Re: Mellanox ConnectX-3 support
«
Reply #4 on:
November 22, 2020, 03:27:55 pm »
That's good to hear. I now have them all disabled.
Any idea how to get the interface to detect the 56gbps speed?
Logged
mimugmail
Hero Member
Posts: 6766
Karma: 494
Re: Mellanox ConnectX-3 support
«
Reply #5 on:
November 22, 2020, 03:35:05 pm »
No idea, maybe ask at freebsd-net mailing list, there are mellanox devs around, maybe some sysctl tweaks
Logged
WWW:
www.routerperformance.net
Support plans:
https://www.max-it.de/en/it-services/opnsense/
Commercial Plugins (German):
https://opnsense.max-it.de/
Print
Pages: [
1
]
« previous
next »
OPNsense Forum
»
Archive
»
20.7 Legacy Series
»
Mellanox ConnectX-3 support