since updating to 22.7 suricate is constantly stopping

[manilx]

Upgraded from 2.7RC2 to 22.67 final.

I now find that suricata is constantly stopping. I turn it on again and after a short while it stops.

Can't find the issue, just upgraded all.

[franco]

You can try to revert, maybe 6.0.6 is having an issue?

# opnsense-revert -r 22.7.r2 suricata

Check dmesg, maybe a segfault or out of memory kill?


Cheers,
Franco

[manilx]

I can revert via hyour command, yes.

Regarding dmesg bear with me but I'm not into cli..... I can paste commands and post them here, no problem.

[manilx]

Reverted but it's still stopping, so this did not fix it.

[manilx]

Here is the dmesg output since last boot:

Code Select Expand

---<<BOOT>>---
Copyright (c) 1992-2021 The FreeBSD Project.
Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994
The Regents of the University of California. All rights reserved.
FreeBSD is a registered trademark of The FreeBSD Foundation.
FreeBSD 13.1-RELEASE stable/22.7-n250212-a26d6065f1f SMP amd64
FreeBSD clang version 13.0.0 (git@github.com:llvm/llvm-project.git llvmorg-13.0.0-0-gd7b669b3a303)
VT(vga): text 80x25
CPU: Intel(R) Core(TM) i7-8550U CPU @ 1.80GHz (1992.09-MHz K8-class CPU)
  Origin="GenuineIntel"  Id=0x806ea  Family=0x6  Model=0x8e  Stepping=10
  Features=0x1f83fbff<FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,MMX,FXSR,SSE,SSE2,SS,HTT>
  Features2=0xfffab223<SSE3,PCLMULQDQ,VMX,SSSE3,FMA,CX16,PDCM,PCID,SSE4.1,SSE4.2,x2APIC,MOVBE,POPCNT,TSCDLT,AESNI,XSAVE,OSXSAVE,AVX,F16C,RDRAND,HV>
  AMD Features=0x2c100800<SYSCALL,NX,Page1GB,RDTSCP,LM>
  AMD Features2=0x121<LAHF,ABM,Prefetch>
  Structured Extended Features=0x9c47ab<FSGSBASE,TSCADJ,BMI1,AVX2,SMEP,BMI2,ERMS,INVPCID,MPX,RDSEED,ADX,SMAP,CLFLUSHOPT>
  Structured Extended Features2=0x4<UMIP>
  Structured Extended Features3=0xac000400<MD_CLEAR,IBPB,STIBP,ARCH_CAP,SSBD>
  XSAVE Features=0xf<XSAVEOPT,XSAVEC,XINUSE,XSAVES>
  IA32_ARCH_CAPS=0x48<SKIP_L1DFL_VME>
  AMD Extended Feature Extensions ID EBX=0x100d000<IBPB,IBRS,STIBP,SSBD>
  VT-x: PAT,HLT,MTF,PAUSE,EPT,UG,VPID
Hypervisor: Origin = "KVMKVMKVM"
real memory  = 6442450944 (6144 MB)
avail memory = 6188003328 (5901 MB)
Event timer "LAPIC" quality 600
ACPI APIC Table: <BOCHS  BXPC    >
FreeBSD/SMP: Multiprocessor System Detected: 6 CPUs
FreeBSD/SMP: 1 package(s) x 6 core(s)
random: registering fast source Intel Secure Key RNG
random: fast provider: "Intel Secure Key RNG"
random: unblocking device.
ioapic0 <Version 1.1> irqs 0-23
Launching APs: 3 4 5 1 2
wlan: mac acl policy registered
random: entropy device external interface
kbd1 at kbdmux0
WARNING: Device "spkr" is Giant locked and may be deleted before FreeBSD 14.0.
vtvga0: <VT VGA driver>
kvmclock0: <KVM paravirtual clock>
Timecounter "kvmclock" frequency 1000000000 Hz quality 975
kvmclock0: registered as a time-of-day clock, resolution 0.000001s
smbios0: <System Management BIOS> at iomem 0xf5980-0xf599e
smbios0: Version: 2.8, BCD Revision: 2.8
aesni0: <AES-CBC,AES-CCM,AES-GCM,AES-ICM,AES-XTS>
acpi0: <BOCHS BXPC>
acpi0: Power Button (fixed)
cpu0: <ACPI CPU> on acpi0
atrtc0: <AT realtime clock> port 0x70-0x77 irq 8 on acpi0
atrtc0: registered as a time-of-day clock, resolution 1.000000s
Event timer "RTC" frequency 32768 Hz quality 0
hpet0: <High Precision Event Timer> iomem 0xfed00000-0xfed003ff on acpi0
Timecounter "HPET" frequency 100000000 Hz quality 950
Timecounter "ACPI-fast" frequency 3579545 Hz quality 900
acpi_timer0: <24-bit timer at 3.579545MHz> port 0x608-0x60b on acpi0
pcib0: <ACPI Host-PCI bridge> port 0xcf8-0xcff on acpi0
pci0: <ACPI PCI bus> on pcib0
isab0: <PCI-ISA bridge> at device 1.0 on pci0
isa0: <ISA bus> on isab0
atapci0: <Intel PIIX3 WDMA2 controller> port 0x1f0-0x1f7,0x3f6,0x170-0x177,0x376,0xe160-0xe16f at device 1.1 on pci0
ata0: <ATA channel> at channel 0 on atapci0
ata1: <ATA channel> at channel 1 on atapci0
uhci0: <Intel 82371SB (PIIX3) USB controller> port 0xe140-0xe15f irq 11 at device 1.2 on pci0
usbus0 on uhci0
usbus0: 12Mbps Full Speed USB v1.0
pci0: <bridge> at device 1.3 (no driver attached)
vgapci0: <VGA-compatible display> mem 0xfd000000-0xfdffffff,0xfea90000-0xfea90fff at device 2.0 on pci0
vgapci0: Boot video device
virtio_pci0: <VirtIO PCI (legacy) Balloon adapter> port 0xe000-0xe03f mem 0xfe400000-0xfe403fff irq 11 at device 3.0 on pci0
vtballoon0: <VirtIO Balloon Adapter> on virtio_pci0
virtio_pci1: <VirtIO PCI (legacy) SCSI adapter> port 0xe040-0xe07f mem 0xfea91000-0xfea91fff,0xfe404000-0xfe407fff irq 10 at device 5.0 on pci0
vtscsi0: <VirtIO SCSI Adapter> on virtio_pci1
virtio_pci2: <VirtIO PCI (legacy) Console adapter> port 0xe080-0xe0bf mem 0xfea92000-0xfea92fff,0xfe408000-0xfe40bfff irq 11 at device 8.0 on pci0
virtio_pci3: <VirtIO PCI (legacy) Network adapter> port 0xe0c0-0xe0ff mem 0xfea93000-0xfea93fff,0xfe40c000-0xfe40ffff irq 10 at device 18.0 on pci0
vtnet0: <VirtIO Networking Adapter> on virtio_pci3
vtnet0: Ethernet address: 2e:80:73:bc:5a:1a
vtnet0: netmap queues/slots: TX 6/256, RX 6/128
000.000765 [ 450] vtnet_netmap_attach       vtnet attached txq=6, txd=256 rxq=6, rxd=128
virtio_pci4: <VirtIO PCI (legacy) Network adapter> port 0xe100-0xe13f mem 0xfea94000-0xfea94fff,0xfe410000-0xfe413fff irq 11 at device 19.0 on pci0
vtnet1: <VirtIO Networking Adapter> on virtio_pci4
vtnet1: Ethernet address: c6:70:b5:f8:da:67
vtnet1: netmap queues/slots: TX 6/256, RX 6/128
000.000766 [ 450] vtnet_netmap_attach       vtnet attached txq=6, txd=256 rxq=6, rxd=128
pcib1: <ACPI PCI-PCI bridge> mem 0xfea95000-0xfea950ff irq 10 at device 30.0 on pci0
pci1: <ACPI PCI bus> on pcib1
pcib2: <ACPI PCI-PCI bridge> mem 0xfea96000-0xfea960ff irq 11 at device 31.0 on pci0
pci2: <ACPI PCI bus> on pcib2
acpi_syscontainer0: <System Container> on acpi0
vmgenc0: <VM Generation Counter> on acpi0
acpi_syscontainer1: <System Container> port 0xaf00-0xaf0b on acpi0
acpi_syscontainer2: <System Container> port 0xafe0-0xafe3 on acpi0
acpi_syscontainer3: <System Container> port 0xae00-0xae17 on acpi0
atkbdc0: <Keyboard controller (i8042)> port 0x60,0x64 irq 1 on acpi0
atkbd0: <AT Keyboard> irq 1 on atkbdc0
kbd0 at atkbd0
atkbd0: [GIANT-LOCKED]
psm0: <PS/2 Mouse> irq 12 on atkbdc0
psm0: [GIANT-LOCKED]
WARNING: Device "psm" is Giant locked and may be deleted before FreeBSD 14.0.
psm0: model IntelliMouse Explorer, device ID 4
fdc0: <floppy drive controller (FDE)> port 0x3f2-0x3f5,0x3f7 irq 6 drq 2 on acpi0
fdc0: does not respond
device_attach: fdc0 attach returned 6
orm0: <ISA Option ROM> at iomem 0xe8000-0xeffff pnpid ORM0000 on isa0
vga0: <Generic ISA VGA> at port 0x3c0-0x3df iomem 0xa0000-0xbffff pnpid PNP0900 on isa0
attimer0: <AT timer> at port 0x40 on isa0
Timecounter "i8254" frequency 1193182 Hz quality 0
Event timer "i8254" frequency 1193182 Hz quality 100
fdc0: No FDOUT register!
Timecounters tick every 10.000 msec
Trying to mount root from ufs:/dev/gpt/rootfs [rw]...
ugen0.1: <Intel UHCI root HUB> at usbus0
uhub0 on usbus0
uhub0: <Intel UHCI root HUB, class 9/0, rev 1.00/1.00, addr 1> on usbus0
da0 at vtscsi0 bus 0 scbus2 target 0 lun 0
da0: <QEMU QEMU HARDDISK 2.5+> Fixed Direct Access SPC-3 SCSI device
da0: 300.000MB/s transfers
da0: Command Queueing enabled
da0: 65536MB (134217728 512 byte sectors)
cd0 at ata1 bus 0 scbus1 target 0 lun 0
cd0: <QEMU QEMU DVD-ROM 2.5+> Removable CD-ROM SCSI device
cd0: Serial Number QM00003
cd0: 16.700MB/s transfers (WDMA2, ATAPI 12bytes, PIO 65534bytes)
cd0: Attempt to query device size failed: NOT READY, Medium not present
uhub0: 2 ports with 2 removable, self powered
vtcon0: <VirtIO Console Adapter> on virtio_pci2
intsmb0: <Intel PIIX4 SMBUS Interface> irq 9 at device 1.3 on pci0
intsmb0: intr IRQ 9 enabled revision 0
smbus0: <System Management Bus> on intsmb0
lo0: link state changed to UP
pflog0: permanently promiscuous mode enabled
vtnet1: link state changed to UP
vtnet0: link state changed to UP
WARNING: attempt to domain_add(netgraph) after domainfinalize()
vtnet1: permanently promiscuous mode enabled
arp: 192.168.2.50 moved from 24:5e:be:64:d0:62 to 24:5e:be:64:d0:63 on vtnet1
arp: 192.168.2.50 moved from 24:5e:be:64:d0:62 to 24:5e:be:64:d0:63 on vtnet1
arp: 192.168.2.50 moved from 24:5e:be:64:d0:63 to 24:5e:be:64:d0:62 on vtnet1
arp: 192.168.2.55 moved from 24:5e:be:5b:e3:ee to 24:5e:be:5b:e3:ef on vtnet1
arp: 192.168.2.50 moved from 24:5e:be:64:d0:62 to 24:5e:be:64:d0:63 on vtnet1
arp: 192.168.2.55 moved from 24:5e:be:5b:e3:ef to 24:5e:be:5b:e3:ee on vtnet1
arp: 192.168.2.55 moved from 24:5e:be:5b:e3:ee to 24:5e:be:5b:e3:ef on vtnet1
root@OPNsense:~ #

[franco]

Hrm, nothing unusual there.

[manilx]

I did one thing after updating:

I changed the interface to be watched from WAN to LAN after reading that since the fw stops all to be stopped anyway.

I changed back to watching the WAN and now suricata does *not* stop. So the interface change was the issue.

Now why does it stop when watching LAN????

[manilx]

p.S. My main firewall is Untangle (for years now).
But I'm trying OPNsense in a proxmox VM for a couple of weeks to see if it fits the bill and for future business installation. So I'm not "at ease" with OPNsense I'm afraid to fix these issues or find the causes.

But I can now confirm 100% that changing from WAN to LAN causes suricata stopping.

[manilx]

I find the following in the suricata log before it stops (when LAN is selected):

Code Select Expand

2022-07-28T16:59:03 Error suricata [101291] <Error> -- [ERRCODE: SC_ERR_NETMAP_CREATE(263)] - opening devname netmap:vtnet1/R failed: Device busy
2022-07-28T16:59:03 Error suricata [101290] <Error> -- [ERRCODE: SC_ERR_NETMAP_CREATE(263)] - opening devname netmap:vtnet1/R failed: Device busy

Any idea on how I can get this working on LAN?

[franco]

What network device name is your WAN?

vtnet(4) sounds like bad luck already but maybe the WAN is vtnet0 which leaves me baffled unless you have something like Zenarmor running on vtnet1 in which case this makes perfect sense. ;)


Cheers,
Franco

[manilx]

WAN is vtnet0. LAN is vtnet1.

Yes, I have Zenarmor running on vtnet1.

[manilx]

Disabling Zenarmor on LAN fixes the issue, Suricata does not stop any longer. So I can't run both on same interface.... Bummer.
Has never been an issue on Untangle.

[franco]

But that's always how it has been for Suricata/Zenarmor in netmap(4) mode on FreeBSD?

You don't need to run IPS mode for Suricata, but then again I am unsure what you are trying to achieve flipping your setup like that against basic recommendation of the services you use.


Cheers,
Franco

[manilx]

Hi,

bear with me as I'm new to OPNsense and probably it's all due to my non-knowledge.

I had Suricata scanning the WAN, with IPS on as I want to block all external unwanted access. Worked like a charm.
I has Zenarmor scanning the LAN so to scan/block the outgoing connections (ad's etc). Worked again fine.

Then I've read that Suricata should be scanning the LAN as the firewall blocks incoming traffic anyway (https://docs.opnsense.org/manual/ips.html#choosing-an-interface), so this is why I switched interfaces and started the issue.

Right now I have reverted to my original working setup (suricata on WAN and zenarmor on LAN), which seemed the logical way to do this on first approach.

[franco]

https://www.sunnyvalley.io/docs/troubleshooting/installation#running-zenarmor-along-with-suricata

"When you use IPS & Zenarmor together, you can only use the WAN interface for Suricata."

So that seems to be what you had and that's good? :)


Cheers,
Franco

(edited this post after reading the sentence again)