Connection lost on all interfaces after boot for few minutes

[QBANIN]

Hi,

Recently I migrated my OPNsense 21.7 config from old box to a new one with latest version installed. It works mostly fine, except that after reboot and beep when all services are up and running suddenly I'm losing all connections to OPNsense. Both LAN and WAN. No ping from both sides, no WebUI, no ssh. After another few minutes connections are restored just like that but even if all services are green some of them are not working correctly, i.e. I have to restart web proxy coz it doesn't pass any traffic, sometimes haproxy returns 503 to all backends even if they're working correclty. The situation gets worse with Surricata enabled but disabling doesn't fix the issue completely.

The interesting part is that if I start pinging LAN before this happens, OPNsense keeps responding. But if I start another ping it doesn't work anymore.

Both my WANs and LANs are connected over VLAns it that changes anything.

My Hardware config:
CPU: i3-N305
RAM: 32 GB
NIC: 6x I226-V

- FC disabled on all interfaces
- ALL hardware offloads and VLAN filtering disabled
- Tried setting net.isr.dispatch=deferred buts seems to get worse with this value
- Messed around with RSS tunables but its completely unusable, HAproxy doesn't work at all

The only messages I found in logs that are somehow connestion-relates is this:

Code: [Select]

<13>1 2024-01-09T00:16:24+01:00 OPNsense.local flowd_aggregate.py 46644 - [meta sequenceId="56"] vacuum done
<13>1 2024-01-09T00:17:00+01:00 OPNsense.local send_telemetry.py 68488 - [meta sequenceId="57"] telemetry data collected 3 records in 0.03 seconds @2024-01-08 23:11:11.555252
<13>1 2024-01-09T00:17:53+01:00 OPNsense.local kernel - - [meta sequenceId="58"] 873.043522 [ 850] iflib_netmap_config       txr 4 rxr 4 txd 1024 rxd 1024 rbufsz 2048
<13>1 2024-01-09T00:17:53+01:00 OPNsense.local kernel - - [meta sequenceId="59"] 873.052058 [ 850] iflib_netmap_config       txr 4 rxr 4 txd 1024 rxd 1024 rbufsz 2048
<13>1 2024-01-09T00:17:53+01:00 OPNsense.local kernel - - [meta sequenceId="60"] 873.060681 [ 850] iflib_netmap_config       txr 4 rxr 4 txd 1024 rxd 1024 rbufsz 2048
<13>1 2024-01-09T00:17:53+01:00 OPNsense.local kernel - - [meta sequenceId="61"] <6>igc0: permanently promiscuous mode enabled
<13>1 2024-01-09T00:17:53+01:00 OPNsense.local kernel - - [meta sequenceId="62"] 873.127056 [ 850] iflib_netmap_config       txr 4 rxr 4 txd 1024 rxd 1024 rbufsz 2048
<13>1 2024-01-09T00:17:53+01:00 OPNsense.local kernel - - [meta sequenceId="63"] <6>igc0: link state changed to DOWN
<13>1 2024-01-09T00:17:53+01:00 OPNsense.local kernel - - [meta sequenceId="64"] <6>igc0_vlan102: link state changed to DOWN
<13>1 2024-01-09T00:17:53+01:00 OPNsense.local kernel - - [meta sequenceId="65"] <6>igc0_vlan101: link state changed to DOWN
<13>1 2024-01-09T00:17:53+01:00 OPNsense.local opnsense 10130 - [meta sequenceId="66"] /usr/local/etc/rc.linkup: DEVD: Ethernet detached event for opt5(igc0_vlan102)
<27>1 2024-01-09T00:17:53+01:00 OPNsense.local dhclient 35942 - [meta sequenceId="67"] connection closed
<26>1 2024-01-09T00:17:53+01:00 OPNsense.local dhclient 35942 - [meta sequenceId="68"] exiting.
<13>1 2024-01-09T00:17:53+01:00 OPNsense.local kernel - - [meta sequenceId="69"] 873.362626 [ 850] iflib_netmap_config       txr 4 rxr 4 txd 1024 rxd 1024 rbufsz 2048
<13>1 2024-01-09T00:17:53+01:00 OPNsense.local kernel - - [meta sequenceId="70"] 873.371161 [ 850] iflib_netmap_config       txr 4 rxr 4 txd 1024 rxd 1024 rbufsz 2048
<13>1 2024-01-09T00:17:53+01:00 OPNsense.local kernel - - [meta sequenceId="71"] 873.379703 [ 850] iflib_netmap_config       txr 4 rxr 4 txd 1024 rxd 1024 rbufsz 2048
<13>1 2024-01-09T00:17:53+01:00 OPNsense.local opnsense 10990 - [meta sequenceId="72"] /usr/local/etc/rc.linkup: DEVD: Ethernet detached event for wan(igc0_vlan101)
<27>1 2024-01-09T00:17:53+01:00 OPNsense.local dhclient 26494 - [meta sequenceId="73"] connection closed
<26>1 2024-01-09T00:17:53+01:00 OPNsense.local dhclient 26494 - [meta sequenceId="74"] exiting.
<13>1 2024-01-09T00:17:53+01:00 OPNsense.local kernel - - [meta sequenceId="75"] 873.616923 [ 850] iflib_netmap_config       txr 4 rxr 4 txd 1024 rxd 1024 rbufsz 2048
<13>1 2024-01-09T00:17:53+01:00 OPNsense.local kernel - - [meta sequenceId="76"] 873.625454 [ 850] iflib_netmap_config       txr 4 rxr 4 txd 1024 rxd 1024 rbufsz 2048
<13>1 2024-01-09T00:17:53+01:00 OPNsense.local kernel - - [meta sequenceId="77"] 873.633988 [ 850] iflib_netmap_config       txr 4 rxr 4 txd 1024 rxd 1024 rbufsz 2048
<13>1 2024-01-09T00:17:53+01:00 OPNsense.local kernel - - [meta sequenceId="78"] <6>igc5: permanently promiscuous mode enabled
<13>1 2024-01-09T00:17:53+01:00 OPNsense.local kernel - - [meta sequenceId="79"] 873.644144 [ 850] iflib_netmap_config       txr 4 rxr 4 txd 1024 rxd 1024 rbufsz 2048
<13>1 2024-01-09T00:17:53+01:00 OPNsense.local kernel - - [meta sequenceId="80"] <6>igc5: link state changed to DOWN
<13>1 2024-01-09T00:17:53+01:00 OPNsense.local kernel - - [meta sequenceId="81"] <6>igc5_vlan1: link state changed to DOWN
<13>1 2024-01-09T00:17:53+01:00 OPNsense.local kernel - - [meta sequenceId="82"] <6>igc5_vlan3: link state changed to DOWN
<13>1 2024-01-09T00:17:53+01:00 OPNsense.local kernel - - [meta sequenceId="83"] <6>igc5_vlan4: link state changed to DOWN
<13>1 2024-01-09T00:17:53+01:00 OPNsense.local kernel - - [meta sequenceId="84"] <6>igc5_vlan99: link state changed to DOWN
<43>1 2024-01-09T00:17:53+01:00 OPNsense.local syslog-ng 6832 - [meta sequenceId="85"] I/O error occurred while writing; fd='39', error='Network is down (50)'
<13>1 2024-01-09T00:17:53+01:00 OPNsense.local kernel - - [meta sequenceId="87"] 873.880694 [ 850] iflib_netmap_config       txr 4 rxr 4 txd 1024 rxd 1024 rbufsz 2048
<13>1 2024-01-09T00:17:53+01:00 OPNsense.local kernel - - [meta sequenceId="88"] 873.889214 [ 850] iflib_netmap_config       txr 4 rxr 4 txd 1024 rxd 1024 rbufsz 2048
<13>1 2024-01-09T00:17:53+01:00 OPNsense.local opnsense 13232 - [meta sequenceId="89"] /usr/local/etc/rc.linkup: DEVD: Ethernet detached event for lan(igc5_vlan1)
<13>1 2024-01-09T00:17:54+01:00 OPNsense.local kernel - - [meta sequenceId="90"] 873.897824 [ 850] iflib_netmap_config       txr 4 rxr 4 txd 1024 rxd 1024 rbufsz 2048
<13>1 2024-01-09T00:17:54+01:00 OPNsense.local opnsense 14048 - [meta sequenceId="91"] /usr/local/etc/rc.linkup: DEVD: Ethernet detached event for opt1(igc5_vlan3)
<13>1 2024-01-09T00:17:54+01:00 OPNsense.local opnsense 15039 - [meta sequenceId="92"] /usr/local/etc/rc.linkup: DEVD: Ethernet detached event for opt2(igc5_vlan4)
<13>1 2024-01-09T00:17:54+01:00 OPNsense.local opnsense 15914 - [meta sequenceId="93"] /usr/local/etc/rc.linkup: DEVD: Ethernet detached event for opt6(igc5_vlan99)
<13>1 2024-01-09T00:17:56+01:00 OPNsense.local kernel - - [meta sequenceId="94"] <6>igc0: link state changed to UP
<13>1 2024-01-09T00:17:56+01:00 OPNsense.local kernel - - [meta sequenceId="95"] <6>igc0_vlan102: link state changed to UP
<13>1 2024-01-09T00:17:56+01:00 OPNsense.local kernel - - [meta sequenceId="96"] <6>igc0_vlan101: link state changed to UP
<13>1 2024-01-09T00:17:56+01:00 OPNsense.local opnsense 18180 - [meta sequenceId="97"] /usr/local/etc/rc.linkup: DEVD: Ethernet attached event for opt5(igc0_vlan102)
<13>1 2024-01-09T00:17:56+01:00 OPNsense.local kernel - - [meta sequenceId="98"] <6>igc5: link state changed to UP
<13>1 2024-01-09T00:17:56+01:00 OPNsense.local kernel - - [meta sequenceId="99"] <6>igc5_vlan1: link state changed to UP
<13>1 2024-01-09T00:17:56+01:00 OPNsense.local kernel - - [meta sequenceId="100"] <6>igc5_vlan3: link state changed to UP
<13>1 2024-01-09T00:17:56+01:00 OPNsense.local kernel - - [meta sequenceId="101"] <6>igc5_vlan4: link state changed to UP
<13>1 2024-01-09T00:17:56+01:00 OPNsense.local kernel - - [meta sequenceId="102"] <6>igc5_vlan99: link state changed to UP
<13>1 2024-01-09T00:18:00+01:00 OPNsense.local opnsense 22295 - [meta sequenceId="103"] /usr/local/etc/rc.routing_configure: ROUTING: entering configure using defaults
<11>1 2024-01-09T00:18:00+01:00 OPNsense.local opnsense 22295 - [meta sequenceId="104"] /usr/local/etc/rc.routing_configure: ROUTING: not a valid interface gateway address: ''
<11>1 2024-01-09T00:18:00+01:00 OPNsense.local opnsense 22295 - [meta sequenceId="105"] /usr/local/etc/rc.routing_configure: ROUTING: not a valid interface gateway address: ''

VLANs 101 and 102 are WANs, the rest are LANs.

and some dmesg

Code: [Select]

Copyright (c) 1992-2021 The FreeBSD Project.
Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994
The Regents of the University of California. All rights reserved.
FreeBSD is a registered trademark of The FreeBSD Foundation.
FreeBSD 13.2-RELEASE-p7 stable/23.7-n254871-d5ec322cffc SMP amd64
FreeBSD clang version 14.0.5 (https://github.com/llvm/llvm-project.git llvmorg-14.0.5-0-gc12386ae247c)
VT(vga): resolution 640x480
CPU: Intel(R) Core(TM) i3-N305 (1804.80-MHz K8-class CPU)
  Origin="GenuineIntel"  Id=0xb06e0  Family=0x6  Model=0xbe  Stepping=0
  Features=0xbfebfbff<FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CLFLUSH,DTS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE>
  Features2=0x7ffafbbf<SSE3,PCLMULQDQ,DTES64,MON,DS_CPL,VMX,EST,TM2,SSSE3,SDBG,FMA,CX16,xTPR,PDCM,PCID,SSE4.1,SSE4.2,x2APIC,MOVBE,POPCNT,TSCDLT,AESNI,XSAVE,OSXSAVE,AVX,F16C,RDRAND>
  AMD Features=0x2c100800<SYSCALL,NX,Page1GB,RDTSCP,LM>
  AMD Features2=0x121<LAHF,ABM,Prefetch>
  Structured Extended Features=0x239ca7eb<FSGSBASE,TSCADJ,BMI1,AVX2,FDPEXC,SMEP,BMI2,ERMS,INVPCID,NFPUSG,PQE,RDSEED,ADX,SMAP,CLFLUSHOPT,CLWB,PROCTRACE,SHA>
  Structured Extended Features2=0x98c007bc<UMIP,PKU,OSPKE,WAITPKG,GFNI,VAES,VPCLMULQDQ,RDPID,MOVDIRI,MOVDIR64B>
  Structured Extended Features3=0xfc184410<FSRM,MD_CLEAR,IBT,IBPB,STIBP,L1DFL,ARCH_CAP,CORE_CAP,SSBD>
  XSAVE Features=0xf<XSAVEOPT,XSAVEC,XINUSE,XSAVES>
  IA32_ARCH_CAPS=0x180fd6b<RDCL_NO,IBRS_ALL,SKIP_L1DFL_VME,MDS_NO,TAA_NO>
  VT-x: PAT,HLT,MTF,PAUSE,EPT,UG,VPID,VID,PostIntr
  TSC: P-state invariant, performance statistics
real memory  = 34358689792 (32767 MB)
avail memory = 33088512000 (31555 MB)
Event timer "LAPIC" quality 600
ACPI APIC Table: <ALASKA A M I >
WARNING: L3 data cache covers more APIC IDs than a package (7 > 3)
FreeBSD/SMP: Multiprocessor System Detected: 8 CPUs
FreeBSD/SMP: 2 package(s) x 4 core(s)
random: registering fast source Intel Secure Key RNG
random: fast provider: "Intel Secure Key RNG"
random: unblocking device.
ioapic0 <Version 2.0> irqs 0-119
Launching APs: 1 3 2 6 7 4 5
random: entropy device external interface
wlan: mac acl policy registered
kbd0 at kbdmux0
WARNING: Device "spkr" is Giant locked and may be deleted before FreeBSD 14.0.
efirtc0: <EFI Realtime Clock>
efirtc0: registered as a time-of-day clock, resolution 1.000000s
smbios0: <System Management BIOS> at iomem 0x75cc5000-0x75cc501e
smbios0: Version: 3.5, BCD Revision: 3.5
aesni0: <AES-CBC,AES-CCM,AES-GCM,AES-ICM,AES-XTS,SHA1,SHA256>
acpi0: <ALASKA A M I >
acpi0: Power Button (fixed)
hpet0: <High Precision Event Timer> iomem 0xfed00000-0xfed003ff on acpi0
Timecounter "HPET" frequency 19200000 Hz quality 950
Event timer "HPET" frequency 19200000 Hz quality 550
attimer0: <AT timer> port 0x40-0x43,0x50-0x53 irq 0 on acpi0
Timecounter "i8254" frequency 1193182 Hz quality 0
Event timer "i8254" frequency 1193182 Hz quality 100
Timecounter "ACPI-fast" frequency 3579545 Hz quality 900
acpi_timer0: <24-bit timer at 3.579545MHz> port 0x1808-0x180b on acpi0
pcib0: <ACPI Host-PCI bridge> port 0xcf8-0xcff on acpi0
pci0: <ACPI PCI bus> on pcib0
vgapci0: <VGA-compatible display> port 0x3000-0x303f mem 0x6000000000-0x6000ffffff,0x4000000000-0x400fffffff at device 2.0 on pci0
vgapci0: Boot video device
xhci0: <XHCI (generic) USB 3.0 controller> mem 0x6001110000-0x600111ffff at device 13.0 on pci0
xhci0: 32 bytes context size, 64-bit DMA
usbus0 on xhci0
usbus0: 5.0Gbps Super Speed USB v3.0
xhci1: <XHCI (generic) USB 3.0 controller> mem 0x6001100000-0x600110ffff at device 20.0 on pci0
xhci1: 32 bytes context size, 64-bit DMA
usbus1 on xhci1
usbus1: 5.0Gbps Super Speed USB v3.0
pci0: <memory, RAM> at device 20.2 (no driver attached)
pci0: <serial bus> at device 21.0 (no driver attached)
pci0: <simple comms> at device 22.0 (no driver attached)
ahci0: <AHCI SATA controller> port 0x3090-0x3097,0x3080-0x3083,0x3060-0x307f mem 0x81700000-0x81701fff,0x81703000-0x817030ff,0x81702000-0x817027ff at device 23.0 on pci0
ahci0: AHCI v1.31 with 1 6Gbps ports, Port Multiplier not supported
ahcich1: <AHCI channel> at channel 1 on ahci0
pcib1: <ACPI PCI-PCI bridge> at device 28.0 on pci0
pci1: <ACPI PCI bus> on pcib1
igc0: <Intel(R) Ethernet Controller I226-V> mem 0x81400000-0x814fffff,0x81500000-0x81503fff at device 0.0 on pci1
igc0: Using 1024 TX descriptors and 1024 RX descriptors
igc0: Using 4 RX queues 4 TX queues
igc0: Using MSI-X interrupts with 5 vectors
igc0: Ethernet address: a8:b8:e0:00:a3:19
igc0: netmap queues/slots: TX 4/1024, RX 4/1024
pcib2: <ACPI PCI-PCI bridge> at device 28.1 on pci0
pci2: <ACPI PCI bus> on pcib2
igc1: <Intel(R) Ethernet Controller I226-V> mem 0x81100000-0x811fffff,0x81200000-0x81203fff at device 0.0 on pci2
igc1: Using 1024 TX descriptors and 1024 RX descriptors
igc1: Using 4 RX queues 4 TX queues
igc1: Using MSI-X interrupts with 5 vectors
igc1: Ethernet address: a8:b8:e0:00:a3:1a
igc1: netmap queues/slots: TX 4/1024, RX 4/1024
pcib3: <ACPI PCI-PCI bridge> at device 28.2 on pci0
pci3: <ACPI PCI bus> on pcib3
igc2: <Intel(R) Ethernet Controller I226-V> mem 0x80e00000-0x80efffff,0x80f00000-0x80f03fff at device 0.0 on pci3
igc2: Using 1024 TX descriptors and 1024 RX descriptors
igc2: Using 4 RX queues 4 TX queues
igc2: Using MSI-X interrupts with 5 vectors
igc2: Ethernet address: a8:b8:e0:00:a3:1b
igc2: netmap queues/slots: TX 4/1024, RX 4/1024
pcib4: <ACPI PCI-PCI bridge> at device 28.3 on pci0
pci4: <ACPI PCI bus> on pcib4
igc3: <Intel(R) Ethernet Controller I226-V> mem 0x80b00000-0x80bfffff,0x80c00000-0x80c03fff at device 0.0 on pci4
igc3: Using 1024 TX descriptors and 1024 RX descriptors
igc3: Using 4 RX queues 4 TX queues
igc3: Using MSI-X interrupts with 5 vectors
igc3: Ethernet address: a8:b8:e0:00:a3:1c
igc3: netmap queues/slots: TX 4/1024, RX 4/1024
pcib5: <ACPI PCI-PCI bridge> at device 28.6 on pci0
pci5: <ACPI PCI bus> on pcib5
igc4: <Intel(R) Ethernet Controller I226-V> mem 0x80800000-0x808fffff,0x80900000-0x80903fff at device 0.0 on pci5
igc4: Using 1024 TX descriptors and 1024 RX descriptors
igc4: Using 4 RX queues 4 TX queues
igc4: Using MSI-X interrupts with 5 vectors
igc4: Ethernet address: a8:b8:e0:00:a3:1d
igc4: netmap queues/slots: TX 4/1024, RX 4/1024
pcib6: <ACPI PCI-PCI bridge> at device 29.0 on pci0
pci6: <ACPI PCI bus> on pcib6
nvme0: <Generic NVMe Device> mem 0x81600000-0x81603fff at device 0.0 on pci6
pcib7: <ACPI PCI-PCI bridge> at device 29.2 on pci0
pci7: <ACPI PCI bus> on pcib7
igc5: <Intel(R) Ethernet Controller I226-V> mem 0x80500000-0x805fffff,0x80600000-0x80603fff at device 0.0 on pci7
igc5: Using 1024 TX descriptors and 1024 RX descriptors
igc5: Using 4 RX queues 4 TX queues
igc5: Using MSI-X interrupts with 5 vectors
igc5: Ethernet address: a8:b8:e0:00:a3:1e
igc5: netmap queues/slots: TX 4/1024, RX 4/1024
isab0: <PCI-ISA bridge> at device 31.0 on pci0
isa0: <ISA bus> on isab0
hdac0: <Intel Alder Lake-N HDA Controller> mem 0x6001120000-0x6001123fff,0x6001000000-0x60010fffff at device 31.3 on pci0
pci0: <serial bus> at device 31.5 (no driver attached)
acpi_button0: <Sleep Button> on acpi0
cpu0: <ACPI CPU> on acpi0
acpi_button1: <Power Button> on acpi0
acpi_tz0: <Thermal Zone> on acpi0
uart0: <16550 or compatible> port 0x3f8-0x3ff irq 4 flags 0x10 on acpi0
uart0: console (115200,n,8,1)
acpi_syscontainer0: <System Container> on acpi0
acpi_syscontainer1: <System Container> on acpi0
atrtc0: <AT realtime clock> at port 0x70 irq 8 on isa0
atrtc0: Warning: Couldn't map I/O.
atrtc0: registered as a time-of-day clock, resolution 1.000000s
Event timer "RTC" frequency 32768 Hz quality 0
hwpstate_intel0: <Intel Speed Shift> on cpu0
hwpstate_intel1: <Intel Speed Shift> on cpu1
hwpstate_intel2: <Intel Speed Shift> on cpu2
hwpstate_intel3: <Intel Speed Shift> on cpu3
hwpstate_intel4: <Intel Speed Shift> on cpu4
hwpstate_intel5: <Intel Speed Shift> on cpu5
hwpstate_intel6: <Intel Speed Shift> on cpu6
hwpstate_intel7: <Intel Speed Shift> on cpu7
Timecounter "TSC" frequency 1804801963 Hz quality 1000
Timecounters tick every 1.000 msec
ZFS filesystem version: 5
ZFS storage pool version: features support (5000)
ugen0.1: <Intel XHCI root HUB> at usbus0
ugen1.1: <Intel XHCI root HUB> at usbus1
uhub0 on usbus0
uhub0: <Intel XHCI root HUB, class 9/0, rev 3.00/1.00, addr 1> on usbus0
nvme0: Allocated 64MB host memory buffer
uhub1 on usbus1
uhub1: <Intel XHCI root HUB, class 9/0, rev 3.00/1.00, addr 1> on usbus1
nvd0: <faspeed P8-128G> NVMe namespace
nvd0: 122104MB (250069680 512 byte sectors)
Trying to mount root from zfs:zroot/ROOT/default []...
uhub0: 3 ports with 3 removable, self powered
uhub1: 16 ports with 16 removable, self powered
igc0: link state changed to UP
igc5: link state changed to UP
ig4iic0: <Intel Alder Lake-M I2C Controller-0> at device 21.0 on pci0
ig4iic0: Using MSI
iicbus0: <Philips I2C bus (ACPI-hinted)> on ig4iic0
iicbus0: <unknown card> at addr 0x40
ichsmb0: <Intel Alder Lake SMBus controller> port 0xefa0-0xefbf mem 0x6001128000-0x60011280ff at device 31.4 on pci0
smbus0: <System Management Bus> on ichsmb0
acpi_wmi0: <ACPI-WMI mapping> on acpi0
acpi_wmi0: cannot find EC device
acpi_wmi0: Embedded MOF found
ACPI: \134_SB.WFDE.WQCC: 1 arguments were passed to a non-method ACPI object (Buffer) (20201113/nsarguments-361)
acpi_wmi1: <ACPI-WMI mapping> on acpi0
acpi_wmi1: cannot find EC device
acpi_wmi1: Embedded MOF found
ACPI: \134_SB.WFTE.WQCC: 1 arguments were passed to a non-method ACPI object (Buffer) (20201113/nsarguments-361)
lo0: link state changed to UP
coretemp0: <CPU On-Die Thermal Sensors> on cpu0
pflog0: permanently promiscuous mode enabled
igc0: link state changed to DOWN
vlan0: changing name to 'igc0_vlan101'
vlan1: changing name to 'igc0_vlan102'
igc5: link state changed to DOWN
vlan2: changing name to 'igc5_vlan1'
vlan3: changing name to 'igc5_vlan3'
vlan4: changing name to 'igc5_vlan4'
vlan5: changing name to 'igc5_vlan99'
igc0: link state changed to UP
igc0_vlan102: link state changed to UP
igc0_vlan101: link state changed to UP
[fib_algo] inet.0 (bsearch4#16) rebuild_fd_flm: switching algo to radix4_lockless
igc5: link state changed to UP
igc5_vlan1: link state changed to UP
igc5_vlan3: link state changed to UP
igc5_vlan4: link state changed to UP
igc5_vlan99: link state changed to UP
wg0: link state changed to UP
tun1: changing name to 'ovpns1'
wg0: link state changed to DOWN
wg0: link state changed to UP
WARNING: attempt to domain_add(netgraph) after domainfinalize()
ipfw2 (+ipv6) initialized, divert loadable, nat loadable, default to accept, logging disabled
load_dn_sched dn_sched FIFO loaded
load_dn_sched dn_sched QFQ loaded
load_dn_sched dn_sched RR loaded
load_dn_sched dn_sched WF2Q+ loaded
load_dn_sched dn_sched PRIO loaded
load_dn_sched dn_sched FQ_CODEL loaded
load_dn_sched dn_sched FQ_PIE loaded
load_dn_aqm dn_aqm CODEL loaded
load_dn_aqm dn_aqm PIE loaded
config_aqm Unable to configure flowset, flowset busy!
config_aqm Unable to configure flowset, flowset busy!
config_aqm Unable to configure flowset, flowset busy!
config_aqm Unable to configure flowset, flowset busy!
config_aqm Unable to configure flowset, flowset busy!

Any clue how to debug and fix this?

[franco]

I guess suricata IPS mode or Zenarmor does that since it needs to hook netmap which will bring down all attached interfaces for technical reasons.


Cheers,
Franco

[QBANIN]

I don't use Zonearmor and this happens with Surricata disabled as well...

[QBANIN]

I guess suricata IPS mode or Zenarmor does that since it needs to hook netmap which will bring down all attached interfaces for technical reasons


Cheers,
Franco

Just noticed that ~10 min connection gap has started just after flowd vacuum process. I soon as I was able to login had to restart squid because it wasnt processing any connection.  (Surricata disabled, no more eth up/down)

Code: [Select]

<13>1 2024-01-09T19:38:04+01:00 OPNsense.local flowd_aggregate.py 74241 - [meta sequenceId="26"] start watching flowd
<13>1 2024-01-09T19:38:05+01:00 OPNsense.local flowd_aggregate.py 74241 - [meta sequenceId="27"] vacuum src_addr_details_086400.sqlite
<13>1 2024-01-09T19:38:08+01:00 OPNsense.local flowd_aggregate.py 74241 - [meta sequenceId="28"] vacuum src_addr_000300.sqlite
<13>1 2024-01-09T19:38:08+01:00 OPNsense.local flowd_aggregate.py 74241 - [meta sequenceId="29"] vacuum src_addr_003600.sqlite
<13>1 2024-01-09T19:38:08+01:00 OPNsense.local flowd_aggregate.py 74241 - [meta sequenceId="30"] vacuum src_addr_086400.sqlite
<13>1 2024-01-09T19:38:08+01:00 OPNsense.local flowd_aggregate.py 74241 - [meta sequenceId="31"] vacuum interface_000030.sqlite
<13>1 2024-01-09T19:38:08+01:00 OPNsense.local flowd_aggregate.py 74241 - [meta sequenceId="32"] vacuum interface_000300.sqlite
<13>1 2024-01-09T19:38:08+01:00 OPNsense.local flowd_aggregate.py 74241 - [meta sequenceId="33"] vacuum interface_003600.sqlite
<13>1 2024-01-09T19:38:08+01:00 OPNsense.local flowd_aggregate.py 74241 - [meta sequenceId="34"] vacuum interface_086400.sqlite
<13>1 2024-01-09T19:38:08+01:00 OPNsense.local flowd_aggregate.py 74241 - [meta sequenceId="35"] vacuum dst_port_000300.sqlite
<13>1 2024-01-09T19:38:08+01:00 OPNsense.local flowd_aggregate.py 74241 - [meta sequenceId="36"] vacuum dst_port_003600.sqlite
<13>1 2024-01-09T19:38:08+01:00 OPNsense.local flowd_aggregate.py 74241 - [meta sequenceId="37"] vacuum dst_port_086400.sqlite
<13>1 2024-01-09T19:38:09+01:00 OPNsense.local flowd_aggregate.py 74241 - [meta sequenceId="38"] vacuum done
<13>1 2024-01-09T19:48:03+01:00 OPNsense.local opnsense 53536 - [meta sequenceId="1"] /usr/local/sbin/pluginctl: plugins_configure webproxy (1,restart)
<165>1 2024-01-09T19:48:23+01:00 OPNsense.local squid 30216 - [meta sequenceId="2"] Squid Parent: squid-1 process 31999 exited due to signal 15 with status 0
<161>1 2024-01-09T19:48:23+01:00 OPNsense.local squid 30216 - [meta sequenceId="3"] Exiting due to unexpected forced shutdown
<165>1 2024-01-09T19:48:30+01:00 OPNsense.local squid 12231 - [meta sequenceId="4"] Squid Parent: will start 1 kids
<165>1 2024-01-09T19:48:30+01:00 OPNsense.local squid 12231 - [meta sequenceId="5"] Squid Parent: (squid-1) process 12654 started