Dell R720 Esxi 6.7 host - Best intell Quad port NiC to use for performance

[sparticle]

Hello,

Hoping someone has a recommendation for an Intel based Quad Port GB Nic card for my Dell R720 Esxi 6.7 host. We run OpnSense on Esxi and the network performance is not great. We have tried all the tweaking that has been posted to try and get OpnSense xBSD using the ESXI VMXNETx adaptors to run at anything approaching wire speed.

All the linux VM's in the host run at wire speed and across the vswitch approaching 10G but the OpnSense VM is not great performance-wise. I reaised issues with the BSD devs but no one has even looked at them. I can also see many other users posting about the network performance issues.

The R720 has a Dell branded Broadcom Quad port card. and I am thinking maybe we switch that to an Intel based card as other posters have said that is the best option as the em driver is the best supported and is in the kernel.

Does anyone have a tested recommendation for an intel based quad port Copper GB card?

Cheers
Spart

[Supermule]

As I said... we run 10gbit/s with X710-T4 nics and they are copper.

[sparticle]

As I said... we run 10gbit/s with X710-T4 nics and they are copper.

Yes, I understand but we do not currently have 10G infrastructure! And we don#t need to pay c. 400 Gbp for a card.

If the Pro 1000PT works its about 40 GBP!

Cheers
Spart

[sparticle]

So does anyone have a tested recommendation for ESXI 6.7 U3 for a quad 1Gb copper Intel based Network adaptor that is performant?

Cheers
Spart

[Patrick M. Hausen]

Sorry, all my appliances have a sufficient number of onboard interfaces.

At 40 £ I would just give it a go - you can always resell it on eBay.

[sparticle]

Sorry, all my appliances have a sufficient number of onboard interfaces.

At 40 £ I would just give it a go - you can always resell it on eBay.

Lots more reading since.

Looks like the R720 can take a daughter card it currently has the Broadcom NetXtreme 5720 based quad port adaptor in it. But can be replaced with an Intel I350 Quad port adaptor.

Shows as fully supported in the latest 13.1 HCL. The BC card is not on the list so maybe that is part of the issue.

I have ordered one and will fit it on the next maintenance reboot of the ESXI server. I will then need to re-config networking on the OpnSense VM as when it boots and now sees an e1000e and not VMXNET3 I will need to PPPoE connection for the wan sorting out.

Cheers
Spart

[cookiemonster]

I hope you get better results but PPoE will still be a limiting factor I think.

[Supermule]

The driver that FBSD sees is the hypervisor presented NIC.

Not the actual one present. So if the problems are related to VmWare and Broadcom drivers, then change the NIC.

Otherwise the issue is elsewhere.

I have 50+ Pro1000 quad cards lying around if you need one.... they are from our old servers.

[sparticle]

The driver that FBSD sees is the hypervisor presented NIC.

Not the actual one present. So if the problems are related to VmWare and Broadcom drivers, then change the NIC.

Otherwise the issue is elsewhere.

I have 50+ Pro1000 quad cards lying around if you need one.... they are from our old servers.

Hey thanks.

I have not issues with network performance on any of the other ESXI VM's they are a combination of Linux and Windows.

But performance on OpnSense/FreeBSD is not great.

I am questioning my choice on replacing the broadcom daughter board with the I350 based one. I have 6 PCI-E 3 slots free 3 full and 3 half height.

What model of PRO1000 are the cards?

I could pop one in and dedicate that to OpnSense.

Cheers
Spart

[cookiemonster]

What is the measure of "performance is not great"? I am not disputing it, but it would be good to measurably understand the baseline and setup.
OP I read "All the linux VM's in the host run at wire speed and across the vswitch approaching 10G but the OpnSense VM is not great performance-wise." wich gives an idea but, what is the setup to improve?
Is it VM to VM on the same host, is there routing involved, where and how? Where does PPoE come into play?
With virtualised setups, there are so many variables that can be at play, maybe it is just me but I don't see the setup in my mind yet.

[sparticle]

What is the measure of "performance is not great"? I am not disputing it, but it would be good to measurably understand the baseline and setup.
OP I read "All the linux VM's in the host run at wire speed and across the vswitch approaching 10G but the OpnSense VM is not great performance-wise." wich gives an idea but, what is the setup to improve?
Is it VM to VM on the same host, is there routing involved, where and how? Where does PPoE come into play?
With virtualised setups, there are so many variables that can be at play, maybe it is just me but I don't see the setup in my mind yet.

I get variable performance across the lan to the OpnSense VM using iperf maxes out at around .6G. As I said all the other VM's stting on the same vswitch run at wire speed all are using the same broadcom physical Quad port NIC.

I have even put the OpnSense LAN on a seperate PG.

There is obviously some issue with the vmx divers. There are many many bug and issue reports to FreeBSD regarding VM performance with teh VMXNET3(x) drivers in ESXI.

Cheers
Spart

Cheers
Spart

[cookiemonster]

Ok so iperf with OPN as server and measuring it from a client to it, not accross it, you get 0.6 Gbps.
On other VMs the same test is almost 10 Gbps.
Got it. I still don't get where PPoE fits but you see now where I'm going. It is very important to describe the issue and how's been measured. Good luck.

[sparticle]

Ok so iperf with OPN as server and measuring it from a client to it, not accross it, you get 0.6 Gbps.
On other VMs the same test is almost 10 Gbps.
Got it. I still don't get where PPoE fits but you see now where I'm going. It is very important to describe the issue and how's been measured. Good luck.

WAN side is PPPoE and would of course have the same reduced performance but as we have <100mb connection not an issue.

[sparticle]

Standard the Dell R720 also supports converged adaptors.

Broadcom 57800T 2 x 10Gbe and 2 x 1Gbe Copper
Intel X540 2 x 10Gbe and 2 x 1 Gbe

So they may also be an option with a forward path if we upgrade the Prosafe switch to include 10Gbe ports.

Does anyone have any experience of ESXI 6.7 and either of those cards with OpnSense they can share?

Cheers
Spart

[johndchch]

Does anyone have any experience of ESXI 6.7 and either of those cards with OpnSense they can share?

not on esxi6 - but I'm running opnsense with an x540-t2 under esxi7 and the performance is great ( driving a 1000/500 fibre connection using pppoe - I have vlan trunking on the 10gbe uplink and split wan/lan out at the vswitch level )