Suricata crashes following upgrade to 16.7

[franco]

FreeBSD never went with the proposed Hyperscan inclusion, that's why it's not a pressing issue there.

The core problem is that Hyperscan requires SSE3 and has no fallbacks without it.

That makes full amd64 compatibility impossible.

I have an idea how to provide support for both, but without FreeBSD requiring this change it will be upon us to introduce it. I don't have an ETA.

There are two workarounds:

1. Use i386 for your hardware. That's a bit of a pain to redo the install.

2. Use the package provided here and "lock" it from the Firmware GUI packages list so that it won't be overwritten on updates. Whenever a new Suricata comes out let us know and we will provide a newer package.


Cheers,
Franco

[franco]

Upon further discussion the CPU should have SSE3, but we've previously seen issues with AMD in general with regard to Hyperscan. We will be keeping a close watch on the issue in any case.

[Manxmann]

Thanks franco,

That sounds like an interesting situation. As per your recommendation I will lock the Suricata package now from updates and call on your generosity should a major update occur that I need to deploy.

With regard to your latter statement yes according to everything I can find it should support SSE3 however looking at a Linux VM running on the same host I get the following, as you can see SSE3 is missing for some reason. I've checked on a couple of DL385G6 servers are the results are the same, I'll start looking to see if I can find any microcode issues/updates on Google:

/proc# cat cpuinfo
processor       : 0
vendor_id       : AuthenticAMD
cpu family      : 16
model           : 8
model name      : Six-Core AMD Opteron(tm) Processor 2431
stepping        : 0
microcode       : 0x10000da
cpu MHz         : 2400.160
cache size      : 512 KB
physical id     : 0
siblings        : 1
core id         : 0
cpu cores       : 1
apicid          : 0
initial apicid  : 0
fpu             : yes
fpu_exception   : yes
cpuid level     : 5
wp              : yes
flags           : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx mmxext fxsr_opt pdpe1gb rdtscp lm 3dnowext 3dnow rep_good nopl extd_apicid pni cx16 x2apic popcnt hypervisor lahf_lm cmp_legacy cr8_legacy abm sse4a misalignsse 3dnowprefetch ibs vmmcall
bogomips        : 4800.32
TLB size        : 1024 4K pages
clflush size    : 64
cache_alignment : 64
address sizes   : 48 bits physical, 48 bits virtual
power management:

[Manxmann]

Silly me, the Opteron Istanbul core does support SSE3 and is shown in the DMESG of OPNSense booting however it doesn't support SSSE3 (extra S). Could this be what HYperscan is using not SSE3?

CPU: Six-Core AMD Opteron(tm) Processor 2431 (2400.14-MHz K8-class CPU)
  Origin="AuthenticAMD"  Id=0x100f80  Family=0x10  Model=0x8  Stepping=0
  Features=0x1783fbff<FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,MMX,FXSR,SSE,SSE2,HTT>
  Features2=0x80a02001<SSE3,CX16,x2APIC,POPCNT,HV>
  AMD Features=0xee500800<SYSCALL,NX,MMX+,FFXSR,Page1GB,RDTSCP,LM,3DNow!+,3DNow!>
  AMD Features2=0x5f3<LAHF,CMP,CR8,ABM,SSE4A,MAS,Prefetch,IBS>

[franco]

You are right. The minimum requirement for Hyperscan is "core2", which added SSSE3 via Intel.

https://github.com/01org/hyperscan/issues/20#issuecomment-218335994

What I don't get is why instruction sets are run during load-time, which lockes out everyone not capable even though Hyperscan wasn't even selected.


Cheers,
Franco

[Jose]

First, I want to thanks the OPNsense team for such great and very responsive firewall software, very happy user here.

Wanted to post in this thread since already related to the OP issue and could be informative to some with similar legacy AMD hardware.

After running OPNsense 16.7.x for a while with outstanding performance results for my relatively old hardware (AMD Athlon 64 X2 5600+, 4GB DDR2, IDE HDD) at home, I decided to start using IPS and followed documentation and only enabled all the "abuse.ch" to start being familiarized with, they were working almost for a week with lots of alerts, until I started reading more about and enabled some "Emerging Threats", after applying the selected ET the service(suricata) started crashing withing seconds after apply and/or restarts.

After struggling with the logs and the "exited on signal 4 (core dumped)" almost for a day, I found this thread and followed post #13 and locked the package from being overwritten as previously denoted and my frustration just ended with a smile,  keep up the good work.

Best regards