[SOLVED] Suricata stops without logging error and won't stay started - ESX 5.5

[ooboyle]

Hi,

Love OPNsense so far and hope to deploy it to 70 sites in the next year but I'm having an impossible time getting Suricata to work. I'm running ESX 5.5 and using e1000 adapters on 3 interfaces.

OPNsense 16.1.14-amd64   
FreeBSD 10.2-RELEASE-p17   
OpenSSL 1.0.2h 3 May 2016
Latest updates are all applied

I've tried with vmxnet3 adapters as well and the service stops immediately. The e1000 adapters allow is to stay on for an hour or so before the service stops. Any change to the WAN interface (including firewall rules) causes Suricata to stop. After a reboot, the Suricata engine starts (as per the log file), but then no message is left when it stops after being left alone for a while.

Any ideas? Any assistance would be greatly appreciated.

Oliver

[phoenix]

Have you disabled the offload functions fo those NICs and how much RAM on the VM?

[ooboyle]

Hi Bill,

Thanks for the quick response!

All hardware offloading is disabled.

There's currently 1GB RAM in this test environment with very little traffic. Is that insufficient?

Oliver

[ooboyle]

To add to this, once the service stops, I need to reboot to get the engine to start again.

[ooboyle]

And now it won't stay on for more than 5 minutes, it seems.

[ooboyle]

I rebooted and added 1GB of RAM (now at 2). The service has remained started for a couple of hours. This is similar to what it did yesterday, though, so I'll report back tomorrow.

[ooboyle]

Interestingly, the service has remained up overnight. Perhaps it was simply a RAM issue.

I'll keep this thread open a little longer before confirming that.

Thanks, Bill. Your comment about RAM may have been the correct track to resolving this one.

[franco]

Hi there,

Interesting, we will add this to the docs. And hope you will report back again with more good news. :)


Cheers,
Franco

[ooboyle]

Hi Franco,

I'm indeed reporting back with good news. The service has remained up since I last posted 5 days ago. This is a test machine and it's only passing my traffic, so we'll have to see if this becomes a moving target with more traffic. Either way, if the suricata service stops with no error at some point, it's likely just missing to RAM. Either disabling rules or adding more RAM should fix the issue.

It would be nice if there was some kind of log error we could rely on however.

Oliver

[franco]

Hi Oliver,

I'll pass that to the Jos for inclusion in the docs, thanks.

And I will try to reproduce this with a sparse VM config to see if the error can be bubbled up properly.


Cheers,
Franco

[ooboyle]

Thanks, Franco. Please contact me if you need more info on my setup to reproduce.

Oliver

[weust]

Very interesting to read this. Am hoping to try out Suricata one day here at home.

Running a VM as well with 1GB. All services but configd are disabled, and it's using 324MB RAM (running the HardenedBSD Exp11 build).
Will try to have a look this weekend by just enabling it.

Btw: Could you see memory increasing over time? If you leave the Dashboard open.
And how much RAM is used at the moment? My guess it's over 1GB usage?

[ooboyle]

Weust,

I wan't using that much RAM either, Right now it seems to be sitting around 41%. When I had 1GB RAM instead of 2GB, it was closer to 80-85%. High, but I didn't think it was alarming. I wasn't seeing any other performance issues.

Unfortunately, I'm having an issue getting SNMP up and running as well so I don't have any historical data at the moment.

Oliver

[weust]

Seems Suricata just needs a lot of RAM.
Not an issue for me, but something to keep in mind.

[ooboyle]

Yep. I got SNMP working yesterday so have some historical data. There doesn't seem to be any noticeable memory leak.