Call for testing: official netmap kernel

[franco]

Yep. Last test kernel, I promise.


Cheers,
Franco

[AceRickslick]

OPNsense 20.7.3-amd64
FreeBSD 12.1-RELEASE-p10-HBSD
OpenSSL 1.1.1g 21 Apr 2020
AMD Athlon 3000G with Radeon Vega Graphics (4 cores)
8GB RAM and Intel E1G42ET Dual Ethernet
Virgin Gig1 Broadband

When downloading large files via an SSL encrypted connection with Sensei 1.6 in standard Routed Mode and the latest 20.7.3 netmap kernel, my download speed maxes out at about 450mbps, but running Speedtest I get about 930mbps. If I change Sensei to Routed Mode with generic netmap driver, SSL downloads peak at 580mbps. If I switch mode to Passive, then SSL downloads reach max speed about 980mbps.

Doing the same SSL download with 20.1 and Sensei in standard Routed Mode I will max out my broadband speed.

Hoping this info can help improve the netmap kernel.

Thanks for the great work being done!

[mb]

Hi AceRickslick,

Thanks for the detailed information. What does ubench tell about your cpu score:

Code Select Expand

ubench -cs

Which ethernet adapter are you using for Sensei-protected interfaces and for the WAN interface?

Can you get 930 Mbps with Sensei on during speedtest test?

[AceRickslick]

Hi mb,

ubench score is 574396.

For Sensei-protected it is igb0 and igb1 is used for WAN. It's an Intel 82576 Chipset ethernet adapter.

Yes, I can get 930 Mbps with Sensei turned on during a speedtest test. Just can't hit those speeds during SSL downloads where before on 20.1 it worked perfectly.

[mb]

Thanks, that's promising.

Can I ask for one more test: what happens if you put Sensei on bypass mode? This will tell if this is related to Sensei or netmap.

[AceRickslick]

When Sensei is in Bypass Mode (Passive Mode), I get full speed on SSL downloads and Speedtest test, about 980 Mbps.

[mb]

Be aware, Passive and Bypass mode are different. Passive mode do not use netmap at all. You can enter Bypass mode via Sensei -> Status -> Enter Bypass Mode

Is it Bypass mode?

[AceRickslick]

My bad it was in Passive Mode, I have retested in Bypass Mode and I get full Gigabit speeds on Speedtest and SSL downloads, about 980 Mbps.

Also just to rule out busy network times/time of day, when I ran the above test, I also tested Sensei in the default Routed Mode, Speedtest hits about 980 Mbps but on the exact same SSL file download as above the speed never gets above 480 Mbps.

[mb]

Hi @Ace,

No worries at all, and thanks for the update. Now it's clear. It looks like we need an optimization on SSL-based classifier.

I'll update the thread once we have some results.

[the-mk]

question: when I try the test kernel with that command - "opnsense-update -kr 20.7.3-netmap" and reboot afterwards, how do I identify with "uname -a" if the "right" kernel (the netmap-test-kernel) is used?

currently I get the following:
FreeBSD OPNsenseTEST.test 12.1-RELEASE-p10-HBSD-FreeBSD 12.1-RELEASE-p10-HBSD #0 517e44a00df(stable/20.7)-dirty: Mon Sep 21 16:21:17 CEST 2020  root@sensey64:/usr/obj/usr/src/amd64.amd64/sys/SMP  amd64

[xpendable]

I just upgraded to the latest 20.7.3 release from 20.1.9 now that this should be the last netmap test kernel.

I disabled Suricata and Sensei first and then proceeded with the upgrade. The upgrade had no issues and the 20.7.3-Netmap kernel installed successfully as well. I then re-enabled Suricata and Sensei, however after everything was back up and running I noticed that my download bandwidth has been reduced considerably as others have mentioned in previous posts as well. Below are my results including general configuration details.

Host: ESXi 6.7 Update 3 (Build 16713306)
OPNsense VM: 4 vCPUs and 8GB RAM
Physcial CPU: Intel Core i7-4790S
Interfaces: vmx0 (LAN - Sensei), vmx1 (WAN - Suricata), no VLANs
Max Download Speed: 750mbps

OPNsense 20.7.3 with 20.7.3-Netmap Kernel:
Suricata Disabled and Sensei PE Stopped (Native) - 720mbps
Suricata Disabled and Sensei PE Bypassed (Native) - 600mbps
Suricata Disabled and Sensei PE Enabled (Native) - 320mbps
Suricata Enabled (Hyperscan/High Profile) and Sensei PE Stopped (Native) - 440mbps
Suricata Enabled (Hyperscan/High Profile) and Sensei PE Bypassed (Native) - 600mbps
Suricata Enabled (Hyperscan/High Profile) and Sensei PE Enabled (Native) - 420mbps

Odd that with Suricata Disabled the throughput goes down, and with Suricata Enabled with the Sensei packet engine bypassed the throughput actually goes up when compared to the stopped state.

OPNsense 20.1.9:
Suricata Enabled and Sensei Enabled - 720mbps

Speedtest was run 3 times in each configuration using the same speedtest server.
I also tried Suricata with the Medium/High & Custom Profile's with no effect on throughput, so I left it on High.

I tested the "generic/emulated" netmap driver through the Sensei config, and the throughput on that was atrocious by dropping the download speed to 15mbps. This is the only setting that also affected my upload throughput.

With Sensei 1.6 my Wireguard interface does not show up as an available interface to add as a protected interface, this should now be possible with Sensei 1.6 correct?

I noticed that once Suricata starts it uses 1 CPU core @ 100% for about 5mins (After Boot as well), during this time you can not run a speedtest as it affects throughput by about 100-150mbps in my setup. Not sure this is relevant, but figured I would mention it as this is new from my OPNsense 20.1.9 install.

[mb]

question: when I try the test kernel with that command - "opnsense-update -kr 20.7.3-netmap" and reboot afterwards, how do I identify with "uname -a" if the "right" kernel (the netmap-test-kernel) is used?

Hi @the-mk, mine shows the following

# opnsense-update -kr 20.7.3-netmap
Your system is up to date.
# uname -a
FreeBSD fw.local 12.1-RELEASE-p10-HBSD FreeBSD 12.1-RELEASE-p10-HBSD #1  ebb8c1489c7(master)-dirty: Mon Sep 21 13:50:27 CEST 2020     root@sensey64:/usr/obj/usr/src/amd64.amd64/sys/SMP  amd64

[mb]

Hi @xpendable, thank you for the detailed test results.

Can you reach out to us via "Report Bug" menu from the Sensei UI? We'd like to have a closer look at your configuration & system.

PS: Make sure you check all options and share relavent information.

[athurdent]

Using Netmap, e.g. by turning on Sensei on LAN or Suricata in IPS mode on WAN, the traffic graph on the dashboard stops working for those interfaces, and Zabbix agent is unable to gather interface statistics, too.

[sy]

Hi @athurdent,
Traffic Graph problem existed with the early 20.7 release but it isn't anymore with 20.7.2 and 3.What is your OPNsense version?