Call for testing: netmap on 20.7

[mb]

Yep, for 1.6 you need the netmap test kernel and you need to be on 20.7. You can try force installing:

pkg add -f os-sensei-1.6.beta1.txz

[bunchofreeds]

OK am running latest Kernel:

12.1-RELEASE-p8-HBSD FreeBSD 12.1-RELEASE-p8-HBSD #6  39e30dc05(master)-dirty: Sat Aug 22 09:35:48 PDT 2020     root@sunnyvalley12.localdomain:/usr/obj/usr/src/amd64.amd64/sys/SMP  amd64

And am running Sensei 1.6 Beta1

I can't see any interfaces in Sensei to apply to. Not even vtnet0 LAN which it is currently running on.

Should I hack in some bypasses https://forum.opnsense.org/index.php?topic=9521.msg84199#msg84199

[mb]

Hi @bunchofreeds,

Sorry, filter was still there. Can you try again:

pkg add -f https://updates.sunnyvalley.io/opnsense/updates/netmap-kernel/os-sensei-1.6.beta1.txz

[bunchofreeds]

@mb,

I now can see the used 'LAN (vtnet0)' interface, but not PPPoE. Only it's parent interface 'Unassigned (vtnet1)'

[heresjody]

Currently have the new kernel installed.

OPNsense 20.7.1-amd64
FreeBSD 12.1-RELEASE-p8-HBSD
OpenSSL 1.1.1g 21 Apr 2020

Code Select Expand

FreeBSD OPNsense.localdomain 12.1-RELEASE-p8-HBSD FreeBSD 12.1-RELEASE-p8-HBSD #6  39e30dc05(master)-dirty: Sat Aug 22 09:35:48 PDT 2020     root@sunnyvalley12.localdomain:/usr/obj/usr/src/amd64.amd64/sys/SMP  amd64

And it seems my surricata is working fine and has started up monitoring PPPoE on my WAN.

Code Select Expand

2020-08-26T18:27:25 suricata[13692] [100200] <Notice> -- all 4 packet processing threads, 4 management threads initialized, engine started.
2020-08-26T18:27:25 suricata[13692] [101008] <Notice> -- opened netmap:pppoe4/T from pppoe4: 0x3e58c16d300
2020-08-26T18:27:25 suricata[13692] [101008] <Notice> -- opened netmap:pppoe4^ from pppoe4^: 0x3e58c16d000
2020-08-26T18:27:25 suricata[13692] [101000] <Notice> -- opened netmap:pppoe4^ from pppoe4^: 0x3e58b442300
2020-08-26T18:27:25 suricata[13692] [101000] <Notice> -- opened netmap:pppoe4/R from pppoe4: 0x3e58b442000
2020-08-26T18:27:25 suricata[13692] [100999] <Notice> -- opened netmap:vtnet0/T from vtnet0: 0x3e58abd4300
2020-08-26T18:27:25 suricata[13692] [100999] <Notice> -- opened netmap:vtnet0^ from vtnet0^: 0x3e58abd4000
2020-08-26T18:27:25 suricata[13692] [100992] <Notice> -- opened netmap:vtnet0^ from vtnet0^: 0x3e587ebc300
2020-08-26T18:27:25 suricata[13692] [100992] <Notice> -- opened netmap:vtnet0/R from vtnet0: 0x3e587ebc000
2020-08-26T18:26:27 suricata[9486] [100971] <Notice> -- This is Suricata version 5.0.3 RELEASE running in SYSTEM mode

If I understand correctly this kernel should fix vtnet-instability with the observed random crashes. Is the code below this line an example of the kind of crash which should now be fixed? (log below is from 20.7.1 with standard kernel and edited for a better reading experience)

Code Select Expand

2020-08-25T10:47:53 kernel 273.390513 [ 320] generic_netmap_register Emulated adapter for ovpnc1 activated
2020-08-25T10:47:53 kernel 273.390098 [1130] generic_netmap_attach Emulated adapter for ovpnc1 created (prev was NULL)
2020-08-25T10:47:53 kernel ovpnc1: permanently promiscuous mode enabled
2020-08-25T10:47:53 kernel 273.385399 [1035] generic_netmap_dtor Emulated netmap adapter for ovpnc1 destroyed
2020-08-25T10:47:53 kernel 273.385329 [1130] generic_netmap_attach Emulated adapter for ovpnc1 created (prev was NULL)
2020-08-25T10:47:53 kernel 273.360774 [ 83] vtnet_free_used 14 sgs dequeued from RX-0 (netmap=1)
2020-08-25T10:47:53 kernel 273.337532 [ 83] vtnet_free_used 15 sgs dequeued from RX-0 (netmap=1)
2020-08-25T10:47:53 kernel 273.313455 [ 83] vtnet_free_used 1 sgs dequeued from TX-0 (netmap=0)

2020-08-25T10:46:54 kernel ---<<BOOT>>---
2020-08-25T10:46:54 syslogd kernel boot file is /boot/kernel/kernel
2020-08-25T10:44:44 syslogd exiting on signal 15
2020-08-25T10:44:42 kernel 082.685532 [ 83] vtnet_free_used 23 sgs dequeued from RX-0 (netmap=1)
2020-08-25T10:44:42 kernel 082.656184 [ 83] vtnet_free_used 127 sgs dequeued from RX-0 (netmap=1)
2020-08-25T10:44:42 kernel 082.656155 [ 83] vtnet_free_used 1 sgs dequeued from TX-0 (netmap=1)
2020-08-25T10:44:42 kernel 082.656113 [1035] generic_netmap_dtor Emulated netmap adapter for ovpnc1 destroyed
2020-08-25T10:44:42 kernel 082.655669 [ 295] generic_netmap_unregister Emulated adapter for ovpnc1 deactivated
2020-08-25T10:44:42 kernel

2020-08-26T17:21:09 kernel 269.933029 [1035] generic_netmap_dtor Emulated netmap adapter for pppoe4 destroyed
2020-08-26T17:21:09 kernel 269.932647 [ 295] generic_netmap_unregister Emulated adapter for pppoe4 deactivated
2020-08-26T17:21:09 kernel 269.745860 [ 320] generic_netmap_register Emulated adapter for pppoe4 activated
2020-08-26T17:21:09 kernel 269.745712 [1130] generic_netmap_attach Emulated adapter for pppoe4 created (prev was NULL)

Update:

It seems Suricata doesn't receive packets from the PPPoE interface. Just changed a setting and this is the output with 0 packets for my PPPoE interface:

Code Select Expand

2020-08-26T18:50:06 suricata[13692] [100200] <Notice> -- Stats for 'pppoe4^': pkts: 0, drop: 0 (nan%), invalid chksum: 0
2020-08-26T18:50:06 suricata[13692] [100200] <Notice> -- Stats for 'pppoe4': pkts: 0, drop: 0 (nan%), invalid chksum: 0
2020-08-26T18:50:06 suricata[13692] [100200] <Notice> -- Stats for 'vtnet0^': pkts: 82103, drop: 0 (0.00%), invalid chksum: 0
2020-08-26T18:50:06 suricata[13692] [100200] <Notice> -- Stats for 'vtnet0': pkts: 74062, drop: 0 (0.00%), invalid chksum: 0

[mb]

Hi @bunchofreeds, Sensei is not meant to be run on WAN. You can test a vpn interface for the tun support.

Speaking of PPPoE and Suricata, we'll revisit this after the first test kernel. Patience :)

Initial goal is to have a "stable" netmap kernel which works flawlessly for the existing drivers.

[mb]

If I understand correctly this kernel should fix vtnet-instability with the observed random crashes. Is the code below this line an example of the kind of crash which should now be fixed? (log below is from 20.7.1 with standard kernel and edited for a better reading experience)

Hi @heresjody, if the firewall crashes and reboots after the messages, yes that is the crash this kernel is fixing.

It seems Suricata doesn't receive packets from the PPPoE interface. Just changed a setting and this is the output with 0 packets for my PPPoE interface:

Since we don't have a pppoe environment we cannot test this on our end, however I'll reach out to @bunchofreeds and yourself and ask to run a test binary. It'll better tell us if netmap is passing packets or not. Theoretically pppoe should work like the  openvpn tun interface.

[bunchofreeds]

@mb,

Thanks for the updates.
Let me know when you have a PPPoE solution that needs testing.

[heresjody]

@mb: Awesome, thanks for the great work. Will be standing by to test your binary.

[loganx1121]

I'm a bit late to the party but I've been experiencing crashes (shown in screenshot attached) every day or 2 on my firewall where it locks up and stops passing traffic.  Between following this thread and github I first installed the sensei 1.6 beta, then per a gentleman on github did the following:

# opnsense-update -kr 20.7.1-netmap4
# opnsense-shell reboot

Firewall is up but now I'm having a weird issue where exceptions in sensei are not working?  For example I have the "Ads" category blocked, but in order to access a site I use I have to whitelist a particular URL which falls into the category.  Having the category blocked but a whitelisted URL in the same category worked fine before I did the above, now it is not. 

I've tried restarting, stopping and starting the sensei engine but it seems like whitelists aren't working anymore if the category is blocked, where as it did before.  Did I screw something up with the kernel update/sensei beta process or is this just a new bug?

[loganx1121]

I'm a bit late to the party but I've been experiencing crashes (shown in screenshot attached) every day or 2 on my firewall where it locks up and stops passing traffic.  Between following this thread and github I first installed the sensei 1.6 beta, then per a gentleman on github did the following:

# opnsense-update -kr 20.7.1-netmap4
# opnsense-shell reboot

Firewall is up but now I'm having a weird issue where exceptions in sensei are not working?  For example I have the "Ads" category blocked, but in order to access a site I use I have to whitelist a particular URL which falls into the category.  Having the category blocked but a whitelisted URL in the same category worked fine before I did the above, now it is not. 

I've tried restarting, stopping and starting the sensei engine but it seems like whitelists aren't working anymore if the category is blocked, where as it did before.  Did I screw something up with the kernel update/sensei beta process or is this just a new bug?

Looks like the kernel recommended by the gentleman on github is different from the test kernel mentioned here.  I've just installed the one from this thread : fetch https://updates.sunnyvalley.io/opnsense/updates/netmap-kernel/kernel-12.1-0822-1.tar.gz

Next I'll try adding the sensei 1.6 beta package from the CLI and will post back with results.

[mb]

Hi @loganx1121,

I doubt kernel is the source for your problem, since exceptions are handled in the sensei packet engine.

Do try 1.6 beta1 and if it does not work out shoot a PR.

Can you point me to the github URL about 20.7.1-netmap4 kernel?

[loganx1121]

Hi @loganx1121,

I doubt kernel is the source for your problem, since exceptions are handled in the sensei packet engine.

Do try 1.6 beta1 and if it does not work out shoot a PR.

Can you point me to the github URL about 20.7.1-netmap4 kernel?

Sure it's here:

https://github.com/opnsense/core/issues/4305

So I uninstalled sensei completely via the GUI and now I'm trying to add the beta but I'm getting the following:

Code Select Expand

root@fw:~ # fetch https://updates.sunnyvalley.io/opnsense/updates/netmap-kernel/os-sensei-1.6.beta1.txz
os-sensei-1.6.beta1.txz                                 25 MB   11 MBps    02s
root@fw:~ # pkg add os-sensei-1.6.beta1.txz
Installing os-sensei-1.6.beta1...
pkg: Missing dependency 'os-sensei-updater'

Failed to install the following 1 package(s): os-sensei-1.6.beta1.txz


And here is the kernel version:

Code Select Expand

root@fw:~ # uname -a
FreeBSD fw 12.1-RELEASE-p8-HBSD FreeBSD 12.1-RELEASE-p8-HBSD #6  39e30dc05(master)-dirty: Sat Aug 22 09:35:48 PDT 2020     root@sunnyvalley12.localdomain:/usr/obj/usr/src/amd64.amd64/sys/SMP  amd64

Any help is appreciated

[mb]

Hi logan, do not completely remove 1.5. just pkg add 1.6; otherwise it'll require dependencies.

[loganx1121]

Ok I'll reinstall sensei from the gui and then add the pkg from cli and report back.  Thanks