Problem with shutdown/reboot as killing suricata gets stuck forever.

[mrzaz]

Hello,
I am running latest 26.1.10 under Unraid VM (QEMU) and a permanent issue that
when doing a reboot or shutdown it is getting stuck trying to kill Suricata forever.

Code Select Expand

root@OPNsense:~ # /usr/local/etc/rc.reboot
>>> Invoking stop script 'beep'
>>> Invoking stop script 'freebsd'
crowdsec_firewall is not running.
Stopping crowdsec.
Waiting for PIDS: 22448.
lldpd not running? (check /var/run/lldpd.pid).
qemu_guest_agent not running? (check /var/run/qemu-ga.pid).
snmpd not running? (check /var/run/net_snmpd.pid).
Stopping suricata.
Waiting for PIDS: 26425

I had it sit for several minutes but still stuck.

I then permanently killed it manually by issuing a separate "kill -9 26425" which then let shutdown to continue.

Code Select Expand

root@OPNsense:~ # /usr/local/etc/rc.reboot
>>> Invoking stop script 'beep'
>>> Invoking stop script 'freebsd'
crowdsec_firewall is not running.
crowdsec not running? (check /var/run/crowdsec_daemon.pid).
lldpd not running? (check /var/run/lldpd.pid).
qemu_guest_agent not running? (check /var/run/qemu-ga.pid).
snmpd not running? (check /var/run/net_snmpd.pid).
Stopping suricata.
Waiting for PIDS: 26425.
Stopping acme_http_challenge.
Waiting for PIDS: 16362.
Stopping flowd.
kill: 6470: No such process
kill: 7055: No such process
Stopping maltrailsensor.
Waiting for PIDS: 91290.
Stopping maltrailserver.
Waiting for PIDS: 88043.
Stopping apcupsd.
kill: 62174: No such process
Stopping flowd_aggregate...done
Stopping monit.
Waiting for PIDS: 85295.
crowdsec not running? (check /var/run/crowdsec_daemon.pid).
crowdsec_firewall is not running.
Stopping tailscaled.
Waiting for PIDS: 44920, 44920.
>>> Invoking stop script 'backup'
>>> Invoking backup script 'captiveportal'
>>> Invoking backup script 'netflow'
>>> Invoking backup script 'rrd'
>>> Invoking stop script 'config'
Shutdown NOW!
shutdown: [pid 90818]

*** FINAL System shutdown message from root@OPNsense.mrzaz.com ***

System going down IMMEDIATELY



*** FINAL System shutdown message from root@OPNsense.mrzaz.com ***

System going down IMMEDIATELY

This is what came in other session where i killed the process

Code Select Expand

root@OPNsense:~ # kill -9 26425
*** FINAL System shutdown message from root@OPNsense.mrzaz.com ***

System going down IMMEDIATELY



*** FINAL System shutdown message from root@OPNsense.mrzaz.com ***

System going down IMMEDIATELY

I have tried this several times at various times and get the same issue everytime. 100% failure.
I do have the "os-qemu-guest-agent" installed/running.

Does anyone else having same issue ?
Any idea of any workaround I could test ?

[franco]

In which mode is suricata running? IDS, IPS (netmap or divert)?


Cheers,
Franco

[cookiemonster]

https://forum.opnsense.org/index.php?topic=52191.msg269089;topicseen#msg269089
Double post or similar but separate problem?

[nero355]

https://forum.opnsense.org/index.php?topic=52191.msg269089;topicseen#msg269089
Double post or similar but separate problem?

This issue is related to that issue as he mentions there two or three times ;)

[mrzaz]

In which mode is suricata running? IDS, IPS (netmap or divert)?


Cheers,
Franco

- Divert (IPS)
Pattern matcher = Aho-Corasick, "Ken Steele" variant.

//Dan Lundqvist

[mrzaz]

https://forum.opnsense.org/index.php?topic=52191.msg269089;topicseen#msg269089
Double post or similar but separate problem?

Yes, I have closed that and keeping this only.
Sorry. Was not sure in which group it was best suited.

//Dan Lundqvist

[mrzaz]

https://forum.opnsense.org/index.php?topic=52191.msg269089;topicseen#msg269089
Double post or similar but separate problem?

This issue is related to that issue as he mentions there two or three times ;)

Hi, that other topic was really not about this issue and I only casually mentioned it as a side-effect.
But it could possible be the main root why it never shut down as it is hanging waiting for suricata PID to stop but never does unless i brutally kill it.

So question is how to proceed. 

I will try what was proposed and we'll see what happens.

>Try running "/usr/local/etc/rc.d/suricata onestop" in the terminal and see what happens and then go and check what's in the "Services -> Intrusion Detection -> >Log File"
>
>/usr/local/etc/rc.d/suricata onestop

//Dan Lundqvist

[mrzaz]

I tried the proposed in other thread.

I get the following in the console:
root@OPNsense:~ # /usr/local/etc/rc.d/suricata onestop
Stopping suricata.
Waiting for PIDS: 71649.
root@OPNsense:~ #


and the following in Suricata Logfile:

2026-06-25T20:47:04  Notice  suricata  [100787] <Notice> -- (W-8000) Verdict: Accepted 0, Dropped 0
2026-06-25T20:47:04  Notice  suricata  [100787] <Notice> -- (W-8000) Treated: Pkts 0, Bytes 0, Errors 0
2026-06-25T20:47:04  Notice  suricata  [100786] <Notice> -- (W-8000) Verdict: Accepted 0, Dropped 0
2026-06-25T20:47:04  Notice  suricata  [100786] <Notice> -- (W-8000) Treated: Pkts 0, Bytes 0, Errors 0
2026-06-25T20:47:04  Notice  suricata  [100785] <Notice> -- (W-8000) Verdict: Accepted 0, Dropped 0
2026-06-25T20:47:04  Notice  suricata  [100785] <Notice> -- (W-8000) Treated: Pkts 0, Bytes 0, Errors 0
2026-06-25T20:47:03  Notice  suricata  [100642] <Notice> -- Signal Received. Stopping engine.

So when doing this, it shuts down but when it is done as the part of OpnSense shutdown it just hangs.
or at least it says that.  It never passes the killing of PID for surcata and never continues with rest
of the shutdown procedure.

//Dan Lundqvist

[franco]

Can you confirm this only happens with divert? It may be an open file descriptor / socket that the kernel doesn't yield.


Cheers,
Franco

[mrzaz]

Can you confirm this only happens with divert? It may be an open file descriptor / socket that the kernel doesn't yield.


Cheers,
Franco

Feels like it happens intermittent now. I tried one more time from console and then it was able to kill all.
Will try one more time from GUI.  Jupp. now that worked as well.

Must be something hanging that is now cleared somehow.
I will monitor this the comming days to see if it re-appers.

//Dan Lundqvist

[mrzaz]

Can you confirm this only happens with divert? It may be an open file descriptor / socket that the kernel doesn't yield.


Cheers,
Franco

If it happens agen  I will try to check that setting.  I prefer to use Divert in the way that is handled.

[franco]

I've looked at the code and it's unclear where Suricata would hang. Has to be in poll() or recvfrom() but both have timeouts and SIGINT/SIGTERM should be properly handled and seen by the application eventually within the span of a second.


Cheers,
Franco

[mrzaz]

I've looked at the code and it's unclear where Suricata would hang. Has to be in poll() or recvfrom() but both have timeouts and SIGINT/SIGTERM should be properly handled and seen by the application eventually within the span of a second.


Cheers,
Franco

Hi Franco,
Thanks for the reply.
It is quite weird. When the issue was present i got the endless waiting for PID suricata that never end. I had to forcefully kill it for the rest of shutdown/reboot stuff to continue. Also when done from gui you didn't know why it was not shutting down or reboot.

I will keep checking if I stumble on it again.

If I do, do you have any commands to print to try to find what is going on?

Best regards
Dan Lundqvist
Stockholm, Sweden