One:
Running IDS mode via PCAP, with buffer modifications
Two:
I've modified the suricata service file to achieve this, not a permanent fix, wanting to discuss and see if it helps others before attempting to see how to integrate this improvement
The whole of my /usr/local/etc/rc.d/suricata:
# Default: root
# suricata_pidfile (str): Pidfile to store pid of suricata process
# Default: /var/run/suricata.pid
. /etc/rc.subr
name="suricata"
rcvar=suricata_enable
start_precmd="suricata_prestart"
command="/usr/local/bin/suricata"
load_rc_config $name
[ -z "$suricata_enable" ] && suricata_enable="NO"
[ -z "$suricata_conf" ] && suricata_conf="/usr/local/etc/suricata/suricata.yaml"
# [ -z "$suricata_flags" ] && suricata_flags="-D"
suricata_flags="-D --pcap-buffer-size=2147483647 -F /usr/local/etc/suricata/capture-filter.bpf"
[ -z "$suricata_divertport" ] && suricata_divertport="8000"
[ -z "$suricata_netmap" ] && suricata_netmap="NO"
[ -z "$suricata_user" ] && suricata_user="root"
[ -z "$suricata_pidfile" ] && suricata_pidfile="/var/run/suricata.pid"
if [ -n "$suricata_interface" ]; then
for interface in $suricata_interface; do
suricata_flags="$suricata_flags --pcap=$interface"
done
elif [ "$suricata_netmap" != "NO" ]; then
suricata_flags="$suricata_flags --netmap"
else
suricata_flags="$suricata_flags -d $suricata_divertport"
info "Inline Mode on divert port $suricata_divertport (suricata_interface not defined)"
fi
pidfile=$suricata_pidfile
suricata_flags="$suricata_flags --pidfile $pidfile"
[ -n "$suricata_conf" ] && suricata_flags="$suricata_flags -c $suricata_conf"
suricata_prestart()
{
if ! run_rc_command status > /dev/null; then
rm -f "$pidfile"
fi
}
run_rc_command "$1"
Three:
I've found that if I watch the interfaces/trunks that have the VLANs I get less drops and I believe this is because of fewer "interfaces", I update/modify this in my custom.yaml in /usr/local/etc/suricata/conf.d/custom.yaml it gets loaded after the suricata.yaml and is a permanent element of OPNSense that can be manually changed
Important elements of my custom.yaml:
host-mode: auto
runmode: autofp
default-packet-size: 1584
max-pending-packets: 25000
defrag:
memcap: 2gb
hash-size: 65536
trackers: 65535
max-frags: 65535
prealloc: yes
timeout: 60
flow:
memcap: 6gb
hash-size: 65536
prealloc: 25600
emergency-recovery: 30
vlan:
use-for-tracking: true
stream:
memcap: 6gb
# memcap-policy: ignore
checksum-validation: no
prealloc-sessions: 262144
#midstream: false
#midstream-policy: ignore
inline: no
bypass: yes
# midstream-policy: bypass
reassembly:
memcap: 1gb
# memcap-policy: bypass
depth: 2mb
toserver-chunk-size: 2560
toclient-chunk-size: 2560
randomize-chunk-size: yes
# randomize-chunk-range: 10
# raw: yes
segment-prealloc: 4096
# check-overlap-different-data: true
mpm-algo: hs
spm-algo: auto
threading:
set-cpu-affinity: yes
detect-thread-ratio: 0.75
pcap:
- interface: ix0
buffer-size: 2147483647
- interface: ix1
buffer-size: 2147483647
- interface: lagg0
buffer-size: 2147483647
With PCAP in Autofp mode and threading:set-cpu-affinity: yes and threading:detect-thread-ratio: 0.75 then at least one or more cores is free to IRQ/etc. and with the BPF I have a three stage filter
BPF Filter lightening the load on the CPUs:
1st group - the VLANs I do not want, including the not element as we are excluding all of this:
not ((ether[12:2] = 0x8100 and (ether[14:2] & 0x0fff = 78 or ether[14:2] & 0x0fff = 234 or ether[14:2] & 0x0fff = 345)) or 2nd group - following the above line (no return/new-line):
(vlan and ((net 192.168.15.0/24 and 192.168.16.0/24) or (net 192.168.15.0/24 and 192.168.89.0/24)) or (net ipv6subnet1 and ipv6subnet2) or (net ipv6subnet1 and ipv6subnet3) or (broadcast or multicast) or (net 192.168.230/24 or 192.168.240.0/24 or 224.0.0.0/4 or ff00::/8 or fe80::/10) or (((icmp or icmp6) or (port 0 or 67 or 68 or 123 or 137 or 138 or 139 or 445 or 514 or 853 or 1900 or 2055 or 2056 or 3000 or 5044 or 5140 or 5142 or 5143 or 5353 or 5355 or 6060 or 8006 or 8291 or 9001 or 9090 or 9200 or 9600)) and (net 192.168.0.0/16 or ipv6subnet::/56)) or (host 192.168.33.44 or 192.168.55.66) or (host ipv6host1 or ipv6host2) or (src host publicip and ((dst port 853) or (src port 00000) or (src portrange 00000-00000 or 00000-00000))) or (dst host publicip and ((src port 853) or (dst port 00000) or (dst portrange 00000-00000 or 00000-00000)))) or 3rd group - following the above line (no return/new-line) is the final and third part (effectively the same as above, only we are say it is not vlan tagged):
(ether[12:2] != 0x8100 and ((net 192.168.15.0/24 and 192.168.16.0/24) or (net 192.168.32.0/24 and 192.168.89.0/24)) or (net ipv6subnet1 and ipv6subnet2) or (net ipv6subnet2 and ipv6subnet3) or (broadcast or multicast) or (net 192.168.230/24 or 192.168.240.0/24 or 224.0.0.0/4 or ff00::/8 or fe80::/10) or (((icmp or icmp6) or (port 0 or 67 or 68 or 123 or 137 or 138 or 139 or 445 or 514 or 853 or 1900 or 2055 or 2056 or 3000 or 5044 or 5140 or 5142 or 5143 or 5353 or 5355 or 6060 or 8006 or 8291 or 9001 or 9090 or 9200 or 9600)) and (net 192.168.0.0/16 or ipv6subnet::/56)) or (host 192.168.33.44 or 192.168.55.66) or (host ipv6host1 or ipv6host2) or (src host publicip and ((dst port 853) or (src port 00000) or (src portrange 00000-00000 or 00000-00000))) or (dst host publicip and ((src port 853) or (dst port 00000) or (dst portrange 00000-00000 or 00000-00000)))))
^ Note about the above bit of BPF, it seems with the Code block when you copy misses the space at the end, the first and second group have a space after the "or" and before the next group, be sure to add it back
Extra edit: The "vlan" element of BPF shifts the inspection point for the host and port elements in the packet, it does not filter in or out VLAN packets, and might be the one place for a logic upgrade to this
If you have enabled stats in your custom.yaml for the eve.json output, this works quite well to watch your drop rate:
tail -f -n 100000 /var/log/suricata/eve.json | jq -c 'select(.event_type=="stats") | select(.stats.capture.kernel_packets > 0) | {timestamp: .timestamp, drop_pct: (.stats.capture.kernel_drops / .stats.capture.kernel_packets * 100)}' ;