OPNsense Forum

English Forums => Zenarmor (Sensei) => Topic started by: mihak on October 05, 2020, 05:52:54 am

Title: Sensei throughput cap despite high-performance device
Post by: mihak on October 05, 2020, 05:52:54 am
I am running OPNSense on a dedicated i7 CPU with 32 GB of memory and 6 gbps ports. Ubench CPU 1132791 and Ubench MEM 2337171. My internet connection is 1 gpbs.

Before installing and enabling Sensei, average throughput on fast.com or speedtest.com was close to 1.0 gbps with a usual overhead penalty. But when Sensei is installed and active on LAN ports (L3 mode with either native or generic nmap driver), throughput drops to 250 mbps - mere 25% of available bandwidth. CPU is idling and never goes above 15%.

I installed the new 20.7.3-netmap driver - but that didn't change the throughput at all.

What am I doing wrong? What troubleshooting data would you like to see?
Title: Re: Sensei throughput cap despite high-performance device
Post by: mr.yx on October 05, 2020, 03:26:45 pm
i am just curious, what i7 do you have that gets those scores? did u bench it with the singlecore flag (ubench -sc)?

a modern coffeelake with 4.2ghz gets arround 690k on single...
Title: Re: Sensei throughput cap despite high-performance device
Post by: mb on October 05, 2020, 05:22:19 pm
Hi @mihak,

We received your report. It looks like 1M cpu score is multi-core score, since single core cpu score seems around 270.000.

Anyhow, I believe you should still be able to attain higher speeds. We'll follow up with you.
Title: Re: Sensei throughput cap despite high-performance device
Post by: mihak on October 06, 2020, 12:16:42 am
SVN team did some analysis on my router today and:

- confirmed that throughput indeed drops from >900 Mbps to ~250 Mbps when Sensei is on and active
- throughput goes back to >900 Mbps when Sensei is in bypass mode
- disabling the hyperthreading of firewall increased the throughput to ~350 Mbps

My device is one of the new-generation Protectli clones: https://www.aliexpress.com/item/4000803229693.html (https://www.aliexpress.com/item/4000803229693.html)

i7 CPU with 32 GB ram and 500 GB mSATA

Will post an update once we progress more.
Title: Re: Sensei throughput cap despite high-performance device
Post by: almodovaris on December 06, 2020, 06:29:09 am
On my APU2C4 (4 cores AMD at 1000 MHz) I have now 356 Mbps to my own provider, through Sensei.

As told by Gary7, I have applied the following tweaks in /boot/loader.conf.local :

vm.pmap.pti = 0
hw.ibrs_disable = 0
Title: Re: Sensei throughput cap despite high-performance device
Post by: Anael on December 14, 2020, 06:55:08 pm
SVN team did some analysis on my router today and:

- confirmed that throughput indeed drops from >900 Mbps to ~250 Mbps when Sensei is on and active
- throughput goes back to >900 Mbps when Sensei is in bypass mode
- disabling the hyperthreading of firewall increased the throughput to ~350 Mbps

My device is one of the new-generation Protectli clones: https://www.aliexpress.com/item/4000803229693.html (https://www.aliexpress.com/item/4000803229693.html)

i7 CPU with 32 GB ram and 500 GB mSATA

Will post an update once we progress more.
Is that a 7th generation proc ? I have an i5 8th and it seems that i got more physical CPUs.  :o
Title: Re: Sensei throughput cap despite high-performance device
Post by: ittk on December 14, 2020, 07:26:08 pm
SVN team did some analysis on my router today and:

- confirmed that throughput indeed drops from >900 Mbps to ~250 Mbps when Sensei is on and active
- throughput goes back to >900 Mbps when Sensei is in bypass mode
- disabling the hyperthreading of firewall increased the throughput to ~350 Mbps

My device is one of the new-generation Protectli clones: https://www.aliexpress.com/item/4000803229693.html (https://www.aliexpress.com/item/4000803229693.html)

i7 CPU with 32 GB ram and 500 GB mSATA

Will post an update once we progress more.
Is that a 7th generation proc ? I have an i5 8th and it seems that i got more physical CPUs.  :o
As far as i know, opnsense and sensei are not capable of using full cpu Power when you have multi-core or Multi threded CPUs. Only Single core with high MHz will get Performance, but not when there are multiple cores. Also Network stack is lacking Features like RSS and PCBGROUP kernel options with supoorting NIC Adapter  and so on to gain full Power out of your modern hardware...

https://forum.opnsense.org/index.php?topic=19420.0
Title: Re: Sensei throughput cap despite high-performance device
Post by: alexroz on December 16, 2020, 07:09:05 pm
Do I get it right - according to https://docs.opnsense.org/vendor/sunnyvalley/sensei_hardwarerequirements.html#cpu-memory Sensei cant provide bandwidth above 500 Mbps?
Title: Re: Sensei throughput cap despite high-performance device
Post by: mb on December 16, 2020, 07:32:04 pm

Hi @ittk, thanks for the pointer. Yes, Sensei, as of now, will be able utilize a single core "per interface".
If you have multiple interfaces, those will be assigned to seperate worker processes, thus seperate CPU cores.

To our experience so far, a vast majority of the current Sensei user base has a WAN connection either 1Gbps or
lower. So our focus has been to deliver a good experience for this segment of the user base.

Work is ongoing for multi-core support per interface.

@alexroz, that chart is a bit conservative :) We'll post an update to the relavent documentation based on the latest figures from the field.

For 1Gbps, a modern CPU (i3 or later) with an ubench score higher than 300.000 should be enough to attain 1Gbps.
I have an Intel i5-5300U CPU @ 2.30GHz at home and can do 1Gbps with AT&T uplink.

Those who are in need of multi-gigabit speeds with Sensei; our experience has been that below CPU is one
of those which is able to provide around 3Gbps throughput using a single CPU core:

https://www.cpubenchmark.net/cpu.php?cpu=Intel+Xeon+E3-1285+v6+%40+4.10GHz&id=3158
Title: Re: Sensei throughput cap despite high-performance device
Post by: alexroz on December 16, 2020, 07:35:55 pm
I am running OPNSense on a dedicated i7 CPU with 32 GB of memory and 6 gbps ports. Ubench CPU 1132791 and Ubench MEM 2337171. My internet connection is 1 gpbs.

Before installing and enabling Sensei, average throughput on fast.com or speedtest.com was close to 1.0 gbps with a usual overhead penalty. But when Sensei is installed and active on LAN ports (L3 mode with either native or generic nmap driver), throughput drops to 250 mbps - mere 25% of available bandwidth. CPU is idling and never goes above 15%.

I installed the new 20.7.3-netmap driver - but that didn't change the throughput at all.

What am I doing wrong? What troubleshooting data would you like to see?

Have anyone tried to apply the following hardware performance optimization technique with opnsense?
https://teklager.se/en/knowledge-base/opnsense-performance-optimization/  ?
Title: Re: Sensei throughput cap despite high-performance device
Post by: almodovaris on December 19, 2020, 08:01:15 pm
Yup, I have applied those tweaks. They are APU specific. But it is wise to get inspired from them and see what works for you. Sensei speed is unlikely to change without changing the kernel.