Call for testing: official netmap kernel

[r4nd0m]

ok, re-installed the kernel again, then sensei and its been stable so far - not sure what it was previously

[mb]

@r4nd0m, thanks for the update.

[r4nd0m]

@r4nd0m, thanks for the update.

found the issue ... looks like the package does not supersede the original package and will be pulled as an upgrade if upgrades are available leading to the boot loop

I suggest this gets another version number eg 20.7.2.1 so the upgrade wont be suggested back to the non patched kernel

https://i.imgur.com/nk5WZSg.png

[nines]

Boot loops are fixed for me now however performance went down from 270mbps to 120mbps while using esxi with vmx.

Sensei is disabled, only using ips

Is this known to be an issue?


Sent from iPhone via Tapatalk

[mb]

Hi @nines, we are able to attain around 1.2-1.3 Gbps between two VMware guests. ubench score around 400K.

What is your HW configuration? What does ubench -cs tell?

[nines]

Hi @mb thanks for your fast response (as always!)

its an esxi 6.7 running on i5-5300U
https://ark.intel.com/content/www/de/de/ark/products/84862/intel-nuc-board-nuc5i5mybe.html

20.1 ubench

Code Select Expand

./ubench -cs
Unix Benchmark Utility v.0.3
Copyright (C) July, 1999 PhysTech, Inc.
Author: Sergei Viznyuk <sv@phystech.com>
http://www.phystech.com/download/ubench.html
FreeBSD 11.2-RELEASE-p20-HBSD FreeBSD 11.2-RELEASE-p20-HBSD  07ef86ce9ca(stable/20.1) amd64
Ubench Single CPU:   420580 (0.40s)

20.7 ubench

Code Select Expand

./ubench -cs
Unix Benchmark Utility v.0.3
Copyright (C) July, 1999 PhysTech, Inc.
Author: Sergei Viznyuk <sv@phystech.com>
http://www.phystech.com/download/ubench.html
FreeBSD 12.1-RELEASE-p9-HBSD FreeBSD 12.1-RELEASE-p9-HBSD  3b652d8ad0e(master) SMP amd64
Ubench Single CPU:   412479 (0.41s)

any thoughts?

[mb]

Hi @nines, all welcome.

CPU score looks good. What happens if you just run nmbridge ? (Both Sensei & Suricata off)

Steps (Assuming vmx0 is your interface)

fetch https://updates.sunnyvalley.io/nmbridge/nmbridge
chmod 750 nmbridge
ifconfig vmx0 -vlanhwtso -vlanhwfilter -vlanhwtag -vlanhwcsum -txcsum -rxcsum -tso4 -tso6 -lro -txcsum6 -rxcsum6
./nmbridge -i netmap:vmx0 -i netmap:vmx0^

[nines]

what would be the expected output?

Code Select Expand

618.451661 main [247] same interface, endpoint 0 goes to host
618.488299 nm_mmap [990] do not mmap, inherit from parent
618.488349 main [269] ------- zerocopy supported
618.488377 main [276] Wait 4 secs for link to come up...
622.499677 main [280] Ready to go, vmx1 0x1/1 <-> vmx1 0x0/1.

//
If I try to do it between vmx0 and vmx1:

Code Select Expand

./nmbridge -i netmap:vmx0 -i netmap:vmx1 the vm instantly becomes unresponsive and the esx console spams "vmx0 drop mbufs that needs checksum offload"

[mb]

Hi Nines,

Do not bridge vmx0 and vmx1, to simulate Suricata/Sensei behaviour bridge the ethernet and the hos operating system.

chmod 750 nmbridge
ifconfig vmx0 -vlanhwtso -vlanhwfilter -vlanhwtag -vlanhwcsum -txcsum -rxcsum -tso4 -tso6 -lro -txcsum6 -rxcsum6
./nmbridge -i netmap:vmx0 -i netmap:vmx0^

[nines]

Hi mb,

Did that also, it's the first quote in my above post but there is no more output other than this.

// during run of nmbridge no traffic forwarding seems to work, ssh is still working though.
if I control+c traffic forwarding continues to work:

Code Select Expand

^C864.502603 main [331] poll timeout [0] ev 1 0 rx 0@42 tx 443, [1] ev 1 0 rx 0@69 tx 511


Sent from iPhone via Tapatalk

[mb]

Did that also, it's the first quote in my above post but there is no more output other than this.

// during run of nmbridge no traffic forwarding seems to work, ssh is still working though.

If you're running nmbridge on vmx1, than you need to disable all offloadings on this interface also:

ifconfig vmx1 -vlanhwtso -vlanhwfilter -vlanhwtag -vlanhwcsum -txcsum -rxcsum -tso4 -tso6 -lro -txcsum6 -rxcsum6
./nmbridge -i netmap:vmx1 -i netmap:vmx1^

Then, run another round of speed test, while nmbridge is running. It is ok that nmbridge won't print much output.

[nines]

thanks for your patience.
its much better but not as good as with 20.1 - now its fluctuating between 130 and 275mbps while 20.1 always peaked at 275 (max isp speed)

[mb]

Hi @nines, no worries at all. My pleasure.

If you can see 275 Mbps now; but not always; I'm inclined to think fluctuations might be related to virtual environment. I'd suggest testing while trying to reduce any side effects imposed by other hosts and/or other services on the firewall.

Reason is, in our test environment, our tests show 1.2 Gbps - 1.3 Gbps between two guests. There's some fluctuation, but it's more like 10% or so.

[nines]

ok, I tested again. Simple http download with disabled suricata/sensei is fast at ~270mbit. If ips is off speeds are max, as soon as its enabled suricata is saturating one core with 100% and speed is capped at 120mbit.

So the reson is definetely something related to ips ...

[mb]

Got it, thanks for the update.