Messages

That seems to have done it, thanks Cedric. Changes made in Notepad, imported without an issue. Saved me a lot of work!

You might have run into this issue:

https://github.com/opnsense/core/issues/9661

Will be fixed in an upcoming release, just make sure when you edit rules you don't use Excel since it can add "" around rows and thats what probably messed your import up.

I was using Excel, perhaps that is it. I can wait for a new release, thanks Cedrik!

I upgraded to 26.1_4 today and tested out the rules migration. After removing some warnings during the import process (mostly interfaces which no longer existed), I attempted to import in the new format.

The import seems to work when done in its entirety, but leaves me with ~180 floating rules, most of which are blank and cannot be removed. I also see warning messages in the system logs but attempting to remove what appears to be an offending interface and reimporting does not help.

Does anyone have any ideas before I resort to recreating my rules manually?

I understand that it is largely out of your hands - but at least know I know where the problem is and can manage it.

Thanks Franco, appreciate your guidance.

Got it, thanks Franco. At this point there is no suggestion that the kernel has or has not had a role in solving the problem.

Your suspicions were correct: as soon as Suricata is set to IPS mode, things start breaking. Suppose I will just have to leave it as is for the time being.

Happy to help if there is any sort of testing needed.

I am certain it's only IPS mode causing this. Have you set a schedule to update rules perhaps?


Cheers,
Franco

Cron job runs at 23:59 to update rules. So far so good with just IDS mode, will test IPS over the weekend and report back.

One question: having installed the kernel patch, I assume this will be reverted with the next minor release and I will need to apply again (if indeed that is part of the fix)?

Thanks for the help so far, much appreciated.

Preliminary feedback: after disabling Suricata and applying the netmap patch, the firewall booted normally without any issues. I do not want to call it resolved yet (since I thought it was resolved yesterday, only to find out not), but it looks positive.

Tomorrow, I shall test another cold boot to see if everything is still healthy. If so, I will enable Suricata in IDS mode, test again, and if everything is still working, revert to IPS mode and test once again.

Will report back in a couple of days with the results, maybe this will help to narrow down the problem.

@Franco let me know if you need anything in particular tested, happy to assist where I can.

Aha, netmap again.. is this Zenarmor or Suricata's doing?

As a general rule DO NOT select VLANs in their settings, only parents and / or try https://forum.opnsense.org/index.php?topic=32114.0


Cheers,
Franco

I caught onto that shortly after posting yeah... for now, I have disabled Suricata and installed the netmap kernel patch (even though I do not see the generic_netmap_register messages). My Suricata config only has LAN selected (igb1). I do not have Zenarmor installed as yet.

Will be doing a cold boot in the next few hours, will report back on situation (latest tomorrow morning).

Thanks Franco, with luck this fixes it.

In the interim - dmesg shows similar logs repeating, not sure if it means anything but it would seem consistent with losing connectivity:

igb1: permanently promiscuous mode disabled
738.093565 [ 851] iflib_netmap_config       txr 2 rxr 2 txd 1024 rxd 1024 rbufsz 2048
igb1: permanently promiscuous mode enabled
738.095867 [ 851] iflib_netmap_config       txr 2 rxr 2 txd 1024 rxd 1024 rbufsz 2048
igb1: link state changed to DOWN
igb1_vlan2: link state changed to DOWN
igb1_vlan3: link state changed to DOWN
igb1_vlan4: link state changed to DOWN
igb1_vlan5: link state changed to DOWN
igb1_vlan6: link state changed to DOWN
igb1_vlan7: link state changed to DOWN
igb1_vlan8: link state changed to DOWN
igb1_vlan9: link state changed to DOWN
igb1_vlan4090: link state changed to DOWN
igb1_vlan10: link state changed to DOWN
igb1_vlan15: link state changed to DOWN
738.374788 [ 851] iflib_netmap_config       txr 2 rxr 2 txd 1024 rxd 1024 rbufsz 2048
igb1: link state changed to UP
igb1_vlan2: link state changed to UP
igb1_vlan3: link state changed to UP
igb1_vlan4: link state changed to UP
igb1_vlan5: link state changed to UP
igb1_vlan6: link state changed to UP
igb1_vlan7: link state changed to UP
igb1_vlan8: link state changed to UP
igb1_vlan9: link state changed to UP
igb1_vlan4090: link state changed to UP
igb1_vlan10: link state changed to UP
igb1_vlan15: link state changed to UP
igb1: permanently promiscuous mode disabled

I can make no other statement without addition information on what is actually not working.


Cheers,
Franco

Understood - I shall try to do a detailed write up later/tomorrow if or when the issue reoccurs. Will try to include as much detail as I can while still (hopefully) making sense.

The log messages have been raised in level from notice to warning. There were no functional changes.


Cheers,
Franco

Thanks for clearing that up Franco. In that case, the messages might be circumstantial rather than an indication of the problem - but there is definitely something wrong, somewhere. That was the only "concerning" thing I could see in the logs. Is there anything else you can think of which might be causing this? Its almost like the interface is being disabled/reenabled constantly, leading to complete outage.

Have you seen the threads relating to certain tunables causing issues that have been hidden until this release? Have you set any additional/custom tunables related to the interface that might be causing a problem?

Yeah I saw those, thanks. Yesterday I defaulted all tunables, which I thought fixed it. Today - no joy. Have reset everything to default again and will test a little later today.

Can I PLEASE get some help with this? Nothing I have tried has made any difference. I thought resetting the tunables to default yesterday fixed it, but apparently not.

I have tried various things but nothing has made any difference. It does seem to come right by itself, eventually, randomly - sometimes it necessitates reloading services via SSH. Changes include allowing all ICMPv6, disabling bogon blocking on WAN, removing dynamic DNS, nothing has helped.

At this point it seems like there is some kind of crash loop with the DHCPv6 server which is also affecting Unbound. Interfaces also seem to be changing state constantly between up and down which is completely breaking the network.

Can anyone help me with this? At this point. I will have no choice but to roll back to 22.7.

After upgrading to 23.1, I have noticed that it takes much longer for inter-VLAN routing to start working when the firewall boots. All of my interface addresses are statically assigned (IPv4 and IPv6) and I have made no configuration changes since upgrading. When looking at the system log, I see messages similar to the below, repeating for about ~3 minutes - during which I have basic ping connectivity to the internet, but no DNS or reachability to anything else inside my network:

/usr/local/etc/rc.linkup: dhcpd_dhcp4_configure() found no suitable IPv4 address on opt7   

This also seems to delay Unbound starting, which would explain my DNS problems as I point to Unbound on the firewall.

Any ideas what might be causing this or what I can check/do to resolve?

Hi Alex,

GUI, Services -> Net-SNMP ->  Display Version in OID. Tick that and poll, it should return the precise version.

The OID is also listed underneath (1.3.6.1.4.1.8072.1.3.2.3.1.2.7.118.101.114.115.105.111.110)