21.7.6 - iflib_netmap_config

[sh00p]

Recently updated to 21.7.6 from 21.7.5 and after doing so I am running into errors.  Initially my LAN would stop routing traffic and I started noticing errors in dmesg and on screen.  I also noticed the same errors when modifying IDS/IPS.  Which made me think it might be the new IDS/Suricata update causing the issue.

I am running:
OPNsense 21.7.6-amd64
FreeBSD 12.1-RELEASE-p21-HBSD
LibreSSL 3.3.5
CPU: AMD Ryzen 5 2400G
NICs: Intel I350-T4
RAM: 16GB

I noticed in dmesg the following:

generic_netmap_dtor        Emulated netmap adapter for igb0_vlan40 destroyed
generic_netmap_attach     Emulated adapter for igb0_vlan40 created (prev was NULL)
generic_netmap_register   Emulated adapter for igb0_vlan40 activated
iflib_netmap_config             txr 4 rxr 4 txd 1024 rxd 1024 rbufsz 2048

After doing some research I have tried a few things.

1. Revert back to 21.7.5 (issue still persisted)
2. Revert suricata (issue still persisted)
3. Revert os-etpro-telemetry (issue still persisted)
4. Reinstalled back to 21.7.6 and all updates.
4. Reinstalled Kernel (issue still persisted)
5. Tuned I350 Gigabit Nic with  (issue still persisted, removed settings):
     dev.igb.0.iflib.override_ntxds="4096"
     dev.igb.1.iflib.override_ntxds="4096"
     dev.igb.0.iflib.override_nrxds="4096"
     dev.igb.1.iflib.override_nrxds="4096"
6. Tried to ensure that intel drivers were loaded during boot with adding if_igb_load="YES" to /boot/loader.conf  (issue still persisted, kept setting)

After all that, the issue still persists, not sure what else to do at this point. Not much information on iflib_netmap_config....  Any suggestions?

[cookiemonster]

Maybe nothing but have you tried changing to OpenSSL instead of LibreSSL ?

[sh00p]

Made the change, LibreSSL to OpenSSL, reboot and the same errors appears.  I get the same errors and eventually it stops routing. 

The only thing I have found that seems to work is not to have IDS/IPS running on interfaces with vlans.  I still get the errors iflib_netmap_config txr 4 rxr 4 txd 1024 rxd 1024 rbufsz 2048.  At this time I just run IDS/IPS on WAN and two other LAN interface with no vlans.

[franco]

It's a bit strange looking at your list: the error always persists no matter what you revert? As for

> 2. Revert suricata (issue still persisted)

Have you made sure to restart the service?

Also running IPS on VLANs is somewhat discouraged and should rather run on the parent interface with the promiscuous mode enbled (it gets rid of netmap emulated adapters).


Cheers,
Franco

[Sparkey]

I received the same thing not routing allowed rebooted same thing a few mins after reboot. Reverted went back to previous version same thing . Re did the entire VM (esx) and same thing occurred on previous version. Leaving IDS  disabled it stable.

Sparkey

[sh00p]

Unless I reverted incorrectly, which I don't think I did, I had rebooted for each change to ensure all services came back and played nicely together..... 

Initially, I was monitoring VLANs, which I know is a no go, but subsequently changed to monitor the parent interface in promiscuous mode.   

As of now, just monitoring interfaces in promiscuous mode is working, but I still get the iflib_netmap_config errors.  (Error:  iflib_netmap_config       txr 4 rxr 4 txd 1024 rxd 1024 rbufsz 2048)

[AdSchellevis]

The iflib_netmap_config message isn't an error, it's just trying to tell you how it initialised netmap.
 
https://github.com/freebsd/freebsd-src/blob/1bfdb812c786ac2607a82633f9c84a5d16f54079/sys/net/iflib.c#L847-L849

https://github.com/freebsd/freebsd-src/commit/21d0c01226eb979556d6d792ec58eb54012fbc24

[Sparkey]

Maybe issue is this?


https://forum.opnsense.org/index.php?topic=25750.15


From Sushifish

Same issue with latest update to 21.7.6 on weekend.
My (internal) interfaces are unreachable. I can log-in through VPN via WAN. After reboot, it runs some time (on Monday full day, Tuesday only 1 h). I've disabled IDS then and it was stable since.
So I suspect an issue with
*ports: suricata 6.0.4[9] with Netmap API version 14 enabled
So internet / WAN interaface is still working, however internal interfaces (on which suricata runs) were unreachable.

[sh00p]

AdSchellevis thanks for this.  After reading this it makes sense.

The iflib_netmap_config message isn't an error, it's just trying to tell you how it initialised netmap.
 
https://github.com/freebsd/freebsd-src/blob/1bfdb812c786ac2607a82633f9c84a5d16f54079/sys/net/iflib.c#L847-L849

https://github.com/freebsd/freebsd-src/commit/21d0c01226eb979556d6d792ec58eb54012fbc24

Also reporting I rolled back suricata (opnsense-revert -r 21.7.5 suricata) and allowed more time to test, it has appeared to resolve my issue, I do continue to see those messages in dmesg.

https://forum.opnsense.org/index.php?topic=25750.15

It would appear these issues might be linked...  I will be keeping my eye on that one.

[AdSchellevis]

When possible, it's also an option to try our beta (22.1) which is based on FreeBSD 13, if it's a driver issue related to netmap, the new version might be different.

Maybe the new netmap api has issues on emulated devices in FreeBSD 12, I haven't seen issues with netmap on physical interfaces on my end.