One:
Running IDS mode via PCAP, with buffer modifications
Two:
I've modified the suricata service file to achieve this, not a permanent fix, wanting to discuss and see if it helps others before attempting to see how to integrate this improvement
The whole of my /usr/local/etc/rc.d/suricata:
Three:
I've found that if I watch the interfaces/trunks that have the VLANs I get less drops and I believe this is because of fewer "interfaces", I update/modify this in my custom.yaml in /usr/local/etc/suricata/conf.d/custom.yaml it gets loaded after the suricata.yaml and is a permanent element of OPNSense that can be manually changed
Important elements of my custom.yaml:
With PCAP in Autofp mode and threading:set-cpu-affinity: yes and threading:detect-thread-ratio: 0.75 then at least one or more cores is free to IRQ/etc. and with the BPF I have a two stage filter (was three, the author learned more about 'vlan' and how it works in BPF, post has since been updated)
The file we will be creating is "/usr/local/etc/suricata/capture-filter.bpf" and it gets used/called with the suricata service file modification listed above
BPF Filter lightening the load on the CPUs:
1st group - the subnets and hosts and port and host combos I do not want, including the not element as we are excluding all of this:
^ Note about the above bit of BPF, it seems with the Code block when you copy misses the space at the end, the first group has a space after the "or" and before the next group, be sure to add it back
Extra edit: Thought I had really dialed it in, my understanding of vlan was a little off, and it works best at the end as it masks all the work after it to shift all subsequent packet payload byte offsets forward by 4 bytes. The original 3 groups are now just two and it appears to work even better. Still in testing to be fair.
Running IDS mode via PCAP, with buffer modifications
Two:
I've modified the suricata service file to achieve this, not a permanent fix, wanting to discuss and see if it helps others before attempting to see how to integrate this improvement
The whole of my /usr/local/etc/rc.d/suricata:
Code Select
# Default: root
# suricata_pidfile (str): Pidfile to store pid of suricata process
# Default: /var/run/suricata.pid
. /etc/rc.subr
name="suricata"
rcvar=suricata_enable
start_precmd="suricata_prestart"
command="/usr/local/bin/suricata"
load_rc_config $name
[ -z "$suricata_enable" ] && suricata_enable="NO"
[ -z "$suricata_conf" ] && suricata_conf="/usr/local/etc/suricata/suricata.yaml"
# [ -z "$suricata_flags" ] && suricata_flags="-D"
suricata_flags="-D --pcap-buffer-size=2147483647 -F /usr/local/etc/suricata/capture-filter.bpf"
[ -z "$suricata_divertport" ] && suricata_divertport="8000"
[ -z "$suricata_netmap" ] && suricata_netmap="NO"
[ -z "$suricata_user" ] && suricata_user="root"
[ -z "$suricata_pidfile" ] && suricata_pidfile="/var/run/suricata.pid"
if [ -n "$suricata_interface" ]; then
for interface in $suricata_interface; do
suricata_flags="$suricata_flags --pcap=$interface"
done
elif [ "$suricata_netmap" != "NO" ]; then
suricata_flags="$suricata_flags --netmap"
else
suricata_flags="$suricata_flags -d $suricata_divertport"
info "Inline Mode on divert port $suricata_divertport (suricata_interface not defined)"
fi
pidfile=$suricata_pidfile
suricata_flags="$suricata_flags --pidfile $pidfile"
[ -n "$suricata_conf" ] && suricata_flags="$suricata_flags -c $suricata_conf"
suricata_prestart()
{
if ! run_rc_command status > /dev/null; then
rm -f "$pidfile"
fi
}
run_rc_command "$1"Three:
I've found that if I watch the interfaces/trunks that have the VLANs I get less drops and I believe this is because of fewer "interfaces", I update/modify this in my custom.yaml in /usr/local/etc/suricata/conf.d/custom.yaml it gets loaded after the suricata.yaml and is a permanent element of OPNSense that can be manually changed
Important elements of my custom.yaml:
Code Select
host-mode: auto
runmode: autofp
default-packet-size: 1584
max-pending-packets: 25000
defrag:
memcap: 2gb
hash-size: 65536
trackers: 65535
max-frags: 65535
prealloc: yes
timeout: 60
flow:
memcap: 6gb
hash-size: 65536
prealloc: 25600
emergency-recovery: 30
vlan:
use-for-tracking: true
stream:
memcap: 6gb
# memcap-policy: ignore
checksum-validation: no
prealloc-sessions: 262144
#midstream: false
#midstream-policy: ignore
inline: no
bypass: yes
# midstream-policy: bypass
reassembly:
memcap: 1gb
# memcap-policy: bypass
depth: 2mb
toserver-chunk-size: 2560
toclient-chunk-size: 2560
randomize-chunk-size: yes
# randomize-chunk-range: 10
# raw: yes
segment-prealloc: 4096
# check-overlap-different-data: true
mpm-algo: hs
spm-algo: auto
threading:
set-cpu-affinity: yes
detect-thread-ratio: 0.75
pcap:
- interface: ix0
buffer-size: 2147483647
- interface: ix1
buffer-size: 2147483647
- interface: lagg0
buffer-size: 2147483647With PCAP in Autofp mode and threading:set-cpu-affinity: yes and threading:detect-thread-ratio: 0.75 then at least one or more cores is free to IRQ/etc. and with the BPF I have a two stage filter (was three, the author learned more about 'vlan' and how it works in BPF, post has since been updated)
The file we will be creating is "/usr/local/etc/suricata/capture-filter.bpf" and it gets used/called with the suricata service file modification listed above
BPF Filter lightening the load on the CPUs:
1st group - the subnets and hosts and port and host combos I do not want, including the not element as we are excluding all of this:
Code Select
not ((ether[12:2] = 0x0800 or ether[12:2] = 0x86DD) and ((net 192.168.15.0/24 and 192.168.16.0/24) or (net 192.168.32.0/24 and 192.168.89.0/24) or (net ipv6subnet1 and ipv6subnet2) or (net ipv6subnet2 and ipv6subnet3) or (broadcast or multicast) or (net 192.168.230/24 or 192.168.240.0/24 or 224.0.0.0/4 or ff00::/8 or fe80::/10) or (((icmp or icmp6) or (port 0 or 67 or 68 or 123 or 137 or 138 or 139 or 445 or 514 or 853 or 1900 or 2055 or 2056 or 3000 or 5044 or 5140 or 5142 or 5143 or 5353 or 5355 or 6060 or 8006 or 8291 or 9001 or 9090 or 9200 or 9600)) and (net 192.168.0.0/16 or ipv6subnet::/56)) or (host 192.168.33.44 or 192.168.55.66) or (host ipv6host1 or ipv6host2) or (src host publicip and ((dst port 853) or (src port 00000) or (src portrange 00000-00000 or 00000-00000))) or (dst host publicip and ((src port 853) or (dst port 00000) or (dst portrange 00000-00000 or 00000-00000))))) or 2nd group - the VLANs I do not want, and a repeat of the exclusions from above, following the above line (no return/new-line):Code Select
((ether[12:2] = 0x8100 or ether[12:2] = 0x88A8 or ether[12:2] = 0x9100) and ((ether[14:2] & 0x0fff = 200 or ether[14:2] & 0x0fff = 300) or (vlan and ((net 192.168.15.0/24 and 192.168.16.0/24) or (net 192.168.15.0/24 and 192.168.89.0/24)) or (net ipv6subnet1 and ipv6subnet2) or (net ipv6subnet1 and ipv6subnet3) or (broadcast or multicast) or (net 192.168.230/24 or 192.168.240.0/24 or 224.0.0.0/4 or ff00::/8 or fe80::/10) or (((icmp or icmp6) or (port 0 or 67 or 68 or 123 or 137 or 138 or 139 or 445 or 514 or 853 or 1900 or 2055 or 2056 or 3000 or 5044 or 5140 or 5142 or 5143 or 5353 or 5355 or 6060 or 8006 or 8291 or 9001 or 9090 or 9200 or 9600)) and (net 192.168.0.0/16 or ipv6subnet::/56)) or (host 192.168.33.44 or 192.168.55.66) or (host ipv6host1 or ipv6host2) or (src host publicip and ((dst port 853) or (src port 00000) or (src portrange 00000-00000 or 00000-00000))) or (dst host publicip and ((src port 853) or (dst port 00000) or (dst portrange 00000-00000 or 00000-00000))))))^ Note about the above bit of BPF, it seems with the Code block when you copy misses the space at the end, the first group has a space after the "or" and before the next group, be sure to add it back
Extra edit: Thought I had really dialed it in, my understanding of vlan was a little off, and it works best at the end as it masks all the work after it to shift all subsequent packet payload byte offsets forward by 4 bytes. The original 3 groups are now just two and it appears to work even better. Still in testing to be fair.
"