Call for testing: netmap on 20.7

[mb]

New 1.6beta3:

(You need to have 1.5.2 or 1.6beta installed)

pkg add -f https://updates.sunnyvalley.io/opnsense/updates/netmap-kernel/os-sensei-1.6.beta3.txz

[scream]

New 1.6beta3:

(You need to have 1.5.2 or 1.6beta installed)

pkg add -f https://updates.sunnyvalley.io/opnsense/updates/netmap-kernel/os-sensei-1.6.beta3.txz

Updated today from 1.6beta1 to 1.6beta3 without any issues.
Now "Web Controls" working fine and it also looks like blocking is working normally on my installation.
So after first 30 minutes I didn't see any issues exept the performance issue already reported with this test kernel and vmx interfaces.


Edit:

Performance:
vmx: 120-130 Mbit/s -> wirespeed is 1gbit/s.
ovpns: 175-213 Mbit/s -> may this is a limit of openvpn encryption.

[DenverTech]

Just adding info from my discussions with support to what Scream was saying. Might save a few people some steps in testing.

- We have 2 sites with latest kernel and 1.6b3, both with 1gbit down and varying upstream. One is VMX and one is igb/ix interfaces (either)
- With the new kernel and 1.6b3, all crashes are gone. It seems to run smoothly...but speed's not ideal.
- Site 1 (ix/igb): With sensei on, we get 20% up/downstream speed. Turn it off, we get 100%. With just bypass, we see about 50%.
- Site 2 (vmx): With sensei on, we get 20% downstream. Upstream is bad at the isp, so not testable. If we turn off Sensei, we get 95%.
- Both sites show about 1% cpu and 10% memory utilization. Doesn't appear to be a load problem.

Will add to the data as I learn more.

[mimugmail]

Here is a FreeBSD upstream discussion if someone is interested:
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=248652

[scream]

Here is a FreeBSD upstream discussion if someone is interested:
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=248652

Thx. Don't know if this really applies to vmx too.

[scream]

Here is a FreeBSD upstream discussion if someone is interested:
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=248652

I had a troubleshooting session with a sensei engineer yesterday.
Created a test FW and two hosts on my esx to test performance between two servers passing opnsense fw.

He confirmed that the bug linked by mimugmail above seems to be the issue with vmx interfaces as well.

The test results on my testsetup looks at follows:

without sensei running at all: 856-935 Mbit/s
with sensei running in bypass mode: 225-405 Mbit/s
with sensei running in normal mode: 205.409 Mbit/s

[heresjody]

Any chance of a post for a netmap kernel for 20.7.2 which includes the cyber crash fix?

@mb: Still willing and able to test PPPoE interface with netmap kernel fixes with Suricata and any test binary you may have. Thanks again for all your hard work.

[mb]

@heresjody, our pleasure. 

We'll post another test kernel based on 20.7.2 early next week. This will -possibly- have additional support for:

• lagg
• bridge

interfaces. We'll get back to pppoe once the first official test kernel gets shipped.

[mr.yx]

Any news on the 1.6 release, 1.5.2_1 with opnsense 20.7.2 mit mlx4en as lan (+several attached vlans to it) is not working. as soon as sensei is enabled all vlan traffic stops, firewall live log shows them as denied, untagged lan is still working.

sensei is running in routed mode (l3), dmesg shows that its using emulated netmap driver.

interface settings: disabled hw crc, tso, lro and hw vlan filtering.

no surricata etc running, as soon as sensei engine is stopped vlans are working again.

i reported this severals days ago via bugreport but ticket is still open.

edit if wrong subforum/thread please move.

[mimugmail]

Isnt it mlx5en? I did some testing with this driver and Connect X3 and it was working (without hardware offloading)

[mr.yx]

its a connectx3 with mlx4en driver loaded. would you mind to show enabled options? ifconfig -m mlx4enX

did you use the beta or stable sensei engine? was it using default netmap or the emulated driver?

thanks in advance

edit2:

Code Select Expand

mlxen0: flags=28943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST,PPROMISC> metric 0 mtu 1500
        options=8c00a8<VLAN_MTU,JUMBO_MTU,VLAN_HWCSUM,VLAN_HWTSO,LINKSTATE>
        capabilities=ed07bb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,TSO4,TSO6,LRO,VLAN_HWFILTER,VLAN_HWTSO,LINKSTATE,RXCSUM_IPV6,TXCSUM_IPV6>
        ether f4:52:14:7a:9b:a0
        inet6 fe80::f652:14ff:fe7a:9ba0%mlxen0 prefixlen 64 scopeid 0x7
        inet 192.168.1.1 netmask 0xffffff00 broadcast 192.168.1.255
        media: Ethernet autoselect (10Gbase-CX4 <full-duplex,rxpause,txpause>)
        status: active
        supported media:
                media autoselect
                media 40Gbase-CR4 mediaopt full-duplex
                media 10Gbase-CX4 mediaopt full-duplex
                media 10Gbase-SR mediaopt full-duplex
                media 1000baseT mediaopt full-duplex
        nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>

also tested it with surricata in ips mode (https://docs.google.com/spreadsheets/u/0/d/1RVj8K3XOzWi-Bkjq6hUxWudu7Cxd8FFTqjLiBMzZWEM/htmlview#gid=0)

same result like active sensei, no vlan traffic/all denied.

Code Select Expand

ifconfig mlxen0 -vlanhwtso has no effect, still enabled afterwards.

Code Select Expand

mlx4_core0@pci0:2:0:0:  class=0x020000 card=0x005515b3 chip=0x100315b3 rev=0x00 hdr=0x00
    vendor     = 'Mellanox Technologies'
    device     = 'MT27500 Family [ConnectX-3]'
    class      = network
    subclass   = ethernet

Code Select Expand

dev.mlx4_core.0.%parent: pci2
dev.mlx4_core.0.%pnpinfo: vendor=0x15b3 device=0x1003 subvendor=0x15b3 subdevice=0x0055 class=0x020000
dev.mlx4_core.0.%location: slot=0 function=0 dbsf=pci0:2:0:0 handle=\_SB_.PCI0.PEG2.PEGP
dev.mlx4_core.0.%driver: mlx4_core
dev.mlx4_core.0.%desc: Mellanox driver (3.5.1)
dev.mlx4_core.%parent:

edit3: maybe related?

https://redmine.pfsense.org/issues/10836

[binaryanomaly]

After upgrading to 20.7.3 you can update to the latest netmap testing kernel with:

Code Select Expand

opnsense-update -kr 20.7.3-netmap

[mb]

Hi @mr.x.y

Can you send the ifconfig mlxen0 command before and after you run below command:

ifconfig mlxen0 -vlanhwtso -vlanhwfilter -vlanhwtag -vlanhwcsum -txcsum -rxcsum -tso4 -tso6 -lro -txcsum6 -rxcsum6

[mb]

Can we continue this thread from here: https://forum.opnsense.org/index.php?topic=19175.new#new . This is more up to date I guess.

[the-mk]

now I have managed to install to 20.7.3-netmap kernel.

after that I logged in on the web gui and did a "check for updates" - and this one shows me that there is an update for the kernel --> current version 20.7.3-netmap vs new version 20.7.3

is it possible that this is because of the different date/time when performing an "uname -a"?

the stock 20.7.3 kernel show something with "Mon Sep 21 16:xx:xx" while the 20.7.3-netmap kernel shows "Mon Sep 21 13:50:27"

the last time I tried to use sensei with the new netmap test kernel - it did not work out good - maybe because I performed an update after installing the -netmap kernel?!?