Problem with shutdown/reboot as killing suricata gets stuck forever.

[mrzaz]

Hi Franco,

I am facing the same issue. I discovered when the system tried to reboot on last update to Business 26.4.1.
My hardware is from Deciso: DEC 697
I had to connect into the console and kill suricata manually as it never rebooted for more than 10 minutes.
My suricata configuration is in divert mode. Since this divert mode became available, I have switched from IPS mode to divert mode as it makes more sense to inspect in suricata only what firewall is allowing, in my case, one specific rule, instead of inspecting all traffic.

I tried the same command showed as before, but the result was always the same. hanging when trying to stop suricata.
I didn't try changing the suricata mode back to IPS or IDS, but as far as I remember, I nave never experienced this hanging issue before. I have been using OpnSense for more than 3 years and this is the first time I encountered this hanging behavior. All previous updates was always smooth with no issues or hanging.

Regards,
Jorge

Hi Jorge,
Then at least I am not alone in this. 🙂

Due to HW constraints in my old opnsense machine I did not use Suricata that much but has now enabled it more and that's when I discovered it.

It always hanged when trying to shutdown. Only thing powerfully enough to kill it was -9.

Dan Lundqvist

[mrzaz]

I have now changed from Divert (IPS) to Netmap (IDS) and let it run for 24-36h and now tried a normal reboot and at least this time it rebooted normally.
Only took a few seconds for suricata PID to stop and continue with rest of the shutdown/reboot.

I will keep this under wrap and test it again in a few days.

If it now is Divert setting that causes it, we need to try to find the culprit.

I will try to revert to Divert (IPS) and see if I could reproduce and then use a bunch of hopefully good commands to debug.

//Dan Lundqvist

[mrzaz]

Hello,
I had it running for 1-2 days using "Netmap (IPS)" and did a few manual reboots and it shuts down/reboot OK.
I then reconfig it to "Divert (IPS)" and then after a while I got the same hard lock.

I did a lot of debug printouts that I could send if someone is interested.
I tried first "kill -TERM 73697" but that did nothing.  process still hanging.
I then did a "kill -9 73697" and then it continued all the way to reboot and then up again.

Seems like it is happening in relative short time on "Divert (IPS)".

//Dan Lundqvist
Stockholm, Sweden

[mrzaz]

Can you confirm this only happens with divert? It may be an open file descriptor / socket that the kernel doesn't yield.


Cheers,
Franco

Hi Franco,
I have now tested with Divert (IPS), then Netmap (IPS) and then back to Divert (IPS) again.

Result is that with "Netmap (IPS)" I do not get this hangings and it could run for hours and
every time I tried a reboot it shutdown clean and restarted as normal.

But when I reverted back to "Divert (IPS) and let it run for a few hours and then try
a reboot it hangs waiting for suricata PID to die. I then tried "kill TERM <pid>"
but did not help.  I then had to go for the big gun with "term -9 <pid>" and then
it continued the shutdown and rebooted OK.

I do have some trace printouts that I could send if someone wants to review ?

//Dan Lundqvist