OPNsense Forum

Archive => 21.7 Legacy Series => Topic started by: FlangeMonkey on September 17, 2021, 03:00:12 pm

Title: ESXi Shaper 1Gb download performance
Post by: FlangeMonkey on September 17, 2021, 03:00:12 pm

Hi Guys,

I have a couple of shapers, one for inbound and one for outbound on my WAN interface.  I am running a virtual firewall, the external WAN interface is now passthrough using igb, which has improved things a lot.

When enabling any shapers using vmxnet3 on the LAN interface, my download performance drops from 800-900Mbit to 600Mbit. 

I therefore changed to E1000e for the LAN, and that increased performance without shapers to between 850-950Mbit.  However, I am still getting a performance drop with shapers to around 700Mbit (during testing, I am increasing the shaper to 1800Mbit to eliminate queue size).

Any Suggestions?

Thanks,

Title: Re: ESXi Shaper 1Gb download performance
Post by: opn_nwo on September 18, 2021, 04:26:06 pm

Somebody correct me if I'm wrong, but I think the shaper is not currently supported in VM setups.

Title: Re: ESXi Shaper 1Gb download performance
Post by: FlangeMonkey on September 21, 2021, 02:53:54 am

I've been using it for years without issues, its just at this speed, its hitting some form of bottle neck.  I know there are card related issues and recommendations for shapers in a virtual machine, such as using e1000 cards over vmxnet3.

I've done a bunch of testing on to try and resolve it and found various things regarding Hz and tickrates on ipfw, I think thats a dead end.  I've ended up dropping in pfsense to see how that performs as a comparison.  That is working very well and although I'm not a fan of pfsense, I'll keep with it until I find the cause.

Edit:  Although I have shapers working in pfsense, the other interesting observation is the performance without shapers, I am noticing a marked performance increase over opnsense.

Title: Re: ESXi Shaper 1Gb download performance
Post by: FlangeMonkey on October 06, 2021, 12:24:44 pm

For anyone else looking, it appears to Spectre and Meltdown Mitigation.  This is a subtle difference between pfSense and opnsense.  I only needed to tweak 'hw.ibrs_disable' to 1, within Tunables.

https://docs.opnsense.org/troubleshooting/hardening.html

Title: Re: ESXi Shaper 1Gb download performance
Post by: dinguz on October 06, 2021, 03:35:22 pm

Check the kern.hz sysctl, in virtualised setups it sometimes gets set to 100 which is too low for traffic shaping to function properly. If this is the case, set it to 1000 and try again.