Messages

1

Intrusion Detection and Prevention / Re: Suricata rule modifications via suricata-update

« on: Today at 08:14:13 am »

In short OPNSense's rule management has got me quite far... but I might be ready for a rather larger logic/control application.

See here:
https://suricata-update.readthedocs.io/en/latest/update.html#modifying-rules

https://suricata-update.readthedocs.io/en/latest/update.html#example-configuration-to-modify-rules-modify-conf

OPNSense has suricata-update already, and if we can safely mod away/quite the OPNSense SID management, and/or, somehow in-between-mod the rule set. Huge win! There are a few hosts that trip rules that if I could filter that rule away from that host, it would be the best win.

On a few rules replacing $HOME_NET with [$HOME_NET, ![192.168.0.20,192.168.0.21]] (for example) would make thing great ^_^

2

Intrusion Detection and Prevention / Re: Suricata rule modifications via suricata-update

« on: Today at 08:07:22 am »

If you were to make a file such as:

Code: [Select]

/usr/local/opnsense/service/conf/actions.d/actions_homelab.conf
You could add this code to it:

Code: [Select]

[configreload]                                                                 
command: /root/suricatamod.sh; exit 0                                           
parameters:                                                                     
type:script                                                                     
message:copy over and reload intrusion detection custom conf                   
description:Copy over and reload intrusion detection custom conf
The shell script would now be able to be scheduled by OPNSense's Web GUI Cron menu

Code: [Select]

/root/suricatamod.shFurther, let's say that script does a thing:

Code: [Select]

#!/bin/sh

# Get current date and time
TIMESTAMP=$(date +"%Y-%m-%d %H:%M:%S")

# Define file paths
ROOT_CUSTOM="/root/custom.yaml"
SURICATA_CUSTOM="/usr/local/etc/suricata/custom.yaml"
SURICATA_TEMPLATE="/usr/local/opnsense/service/templates/OPNsense/IDS/custom.yaml" # do not replace, breaks things
echo "$TIMESTAMP: Checking for configuration updates..." > /root/suricatasame.log

script_name="rule-updater.py"

# Check if the script is running using ps
ps aux | grep "$script_name" | grep -v grep > /dev/null

if [ $? -eq 0 ]; then
  echo "$TIMESTAMP: Script '$script_name' is already running." >> /root/suricatarestart.log
  exit 0 # Exit with a 0
else
  echo "$TIMESTAMP: Script '$script_name' is not running. Proceeding..." >> /root/suricatasame.log
fi

script_name="installRules.py"

# Check if the script is running using ps
ps aux | grep "$script_name" | grep -v grep > /dev/null

if [ $? -eq 0 ]; then
  echo "$TIMESTAMP: Script '$script_name' is already running." >> /root/suricatarestart.log
  exit 0 # Exit with a 0
else
  echo "$TIMESTAMP: Script '$script_name' is not running. Proceeding..." >> /root/suricatasame.log
fi

# Check if files are identical
if cmp -s "$ROOT_CUSTOM" "$SURICATA_CUSTOM"; then
  echo "$TIMESTAMP: Files are identical." >> suricatasame.log
else
  echo "$TIMESTAMP: Files are different, copying $ROOT_CUSTOM to $SURICATA_CUSTOM" >> /root/suricatarestart.log
  cp "$ROOT_CUSTOM" "$SURICATA_CUSTOM"
  # cp "$ROOT_CUSTOM" "$SURICATA_TEMPLATE"
  # Restart Suricata service (adjust command as needed for your system)
  service suricata restart
  echo "$TIMESTAMP: Suricata service restarted." >> /root/suricatarestart.log
fi

exit 0
Now, the result is you get a Suricata installation that can use a (and keeps using) very expressive custom.yaml file, such as this one where I have new vars address-groups and the like:
https://www.nova-labs.net/homelab-opnsense-crowdsec-multi-server/

3

Intrusion Detection and Prevention / Re: Suricata rule modifications via suricata-update

« on: November 20, 2024, 09:12:53 am »

For those that haven't used oinkmaster or pulled-pork, you can more or less replace text and other modifications based on the modify file - now for example, that "specific device" that does something dumb and hits a good rule (but is an accepted risk), you can exclude that specific IP with a modification to the "$HOME_NET" text, and make the rule say "[$HOME_NET, !192.168.0.15]" instead (for that area, keeping all other text in the rule).

This modification functionality does not appear available in OPNSense, but the Policy Management design is okay, but honestly, we can do the same thing with suricata-update and more. Food for thought, curious if anyone else is interested in this?

4

Intrusion Detection and Prevention / Suricata rule modifications via suricata-update

« on: November 20, 2024, 09:03:42 am »

Goal:
To modify specific SIDs using suricata-update's "--modify" option

To do this we need suricata-update, and it is already installed w/Suricata! ^_^ Nice!!

Now, it has a lot of flags/options, and OPNSense's Suricata installation is a bit unique, so, after a bit of discovery I think I've resolve this to be the most accurate form of the command:

Code: [Select]

suricata-update --suricata-conf /usr/local/etc/suricata/suricata.yaml --suricata /usr/local/bin/suricata --data-dir /usr/local/etc/suricata --no-merge --modify-conf=/root/suricata/modify.conf --output /usr/local/etc/suricata/rules --no-test --no-reload --offline
This appears to update/replace files in the /usr/local/suricata/rules folder, but, it does not make a sid map file in the same format (v1 and v2 both look different than the one OPNSense sets up) and OPNSense already does things for IDS Rules in a two different main fashions...

The two fashions are defined in this file:

Code: [Select]

/usr/local/opnsense/service/conf/actions.d/actions_ids.conf
In it are the update and reload details and a few other actions. The update (update) and reload (install) scrips are as such:

Rule Updater by OPNSense

Code: [Select]

/usr/local/opnsense/scripts/suricata/rule-updater.py( gets updated rules from the internet, puts them in /usr/local/etc/suricata/rules )

Rule Installer by OPNSense

Code: [Select]

/usr/local/opnsense/scripts/suricata/installRules.py( gets rules from ? /usr/local/etc/suricata/rules ?, and appears to copy/mod them to /usr/local/etc/suricata/opnsense.rules and then makes /usr/local/etc/suricata/installed_rules.yaml - it seems to also make an SQLite file for the SIDs )

What I'm not sure about is, what might break if I get in the mix here, and...

• Disable OPNSense's Update & Reload (Install) Cron routines
• Setup new Cron Scripts in /usr/local/opnsense/service/conf/actions.d with 'Descriptions' so I can Cron them
• Have these scripts run the suricata-update with the correct flags, and create the expected merged file?

5

General Discussion / Cannot match API getRule state to Firewall Rule in GUI

« on: August 13, 2024, 11:40:03 pm »

In my Syslog output, it seems filterlog/firewall applied a simple numeric to the 'rule' and 'subrule' elements

In the API output, each rule has a UUID and I get them at

Code: [Select]

/diagnostics/firewall/listRuleIds then query the rule's elements at

Code: [Select]

/firewall/filter/getRule?{UUID} and for the output below I've summarized the output as it doesn't seem like the rule data matches what is in the GUI for OPNSense? These are some of the default rules, but, their data all says Pass and that does not make sense to me as the first four are block rules.

Code: [Select]

3c2cd03c70091e3732710e44c3b97506 named Block bogon IPv4 networks from WAN is...
['action pass Pass',
 'direction in In',
 'ipprotocol inet IPv4',
 'protocol any any',
 'gateway  None']
57401c13616c94401fc89cafa777581e named Block bogon IPv6 networks from WAN is...
['action pass Pass',
 'direction in In',
 'ipprotocol inet IPv4',
 'protocol any any',
 'gateway  None']
1072878c6245b52440bc89c6107a9d0a named Block private networks from WAN is...
['action pass Pass',
 'direction in In',
 'ipprotocol inet IPv4',
 'protocol any any',
 'gateway  None']
dcde0621a9f0daa594b014e15f65c076 named Block private networks from WAN is...
['action pass Pass',
 'direction in In',
 'ipprotocol inet IPv4',
 'protocol any any',
 'gateway  None']
5ddcbf1f0688962629f1a2166ba2ab0c named CARP defaults is...
['action pass Pass',
 'direction in In',
 'ipprotocol inet IPv4',
 'protocol any any',
 'gateway  None']
846c09139ef5484c01967052b15e454a named CARP defaults is...
['action pass Pass',
 'direction in In',
 'ipprotocol inet IPv4',
 'protocol any any',
 'gateway  None']
In the end, I am after a method to match the rule+subrule to a UUID, so that I can label my rules in my Syslog output/Kibana so that it shows the firewall histogram w/Rule naming context?

6

Intrusion Detection and Prevention / Re: How do I change suricata.yaml and get it to stick

« on: July 27, 2024, 07:44:13 am »

Generation error message example

7

Intrusion Detection and Prevention / Re: How do I change suricata.yaml and get it to stick

« on: July 27, 2024, 07:39:50 am »

While I do not have a solution, I did want to mention you can edit

Code: [Select]

/usr/local/etc/suricata/custom.yaml and then simply restart the service and have that change be used and stay for a while. You can even replace the "host-os-policy:" area here it seems, and enable additional features in "app-layer" that are normally disabled by default.

If you use the OPNSense IDS Administration GUI, set a Policy, or enable or disable a feature or Rule, the back-end actions will over write your custom.yaml file with the one found at

Code: [Select]

/usr/local/opnsense/service/templates/OPNsense/IDS/custom.yaml and luckily you can modify that file a little and have it work or at least in the past you could - I am currently having some difficulty there.

If you change the /usr/local/opnsense/service/templates/OPNsense/IDS/custom.yaml file at all it appears it will have a generation failure that shows up in the OPNSense IDS Admin GUI. If you delete the /usr/local/opnsense/service/templates/OPNsense/IDS/custom.yaml file, it will have a generation failure.

In short, currently, there is no way to do it.

Default settings are good, I want to customize some XFF output and have that stay around

A supported "custom.yaml" file where ideally you could over-write/replace all and add to suricata.yaml set options (in short you could replace most if not all the existing settings and/or add to them would be amazing.

8

24.7 Production Series / Re: 24.7 upgrade was smooth

« on: July 25, 2024, 11:35:41 pm »

Can confirm!

IPv4 and IPv6 working well, Unbound/CrowdSec/Wireguard working as expected

Have not adopted the new DHCP method(s) yet, still on the old service, to dos, to dos..

Thank you OPNSense!!

9

24.1 Legacy Series / Re: Upgrade to 24.1.2 appears to be downloading a massive base-24.1.2.txz

« on: February 20, 2024, 04:53:50 pm »

It is a rather tiny file of about 100+ Mbs, not sure what is going on...

https://pkg.opnsense.org/FreeBSD:13:amd64/24.1/sets/

Edit:
I wait for about 5 rows of "......." and I manually restarted, asked it to check for updates, and it updated the two base/kernel packages, and everything appears good to go.

Possibly unique hiccup? Not sure, but recoverable it appears!

10

24.1 Legacy Series / Upgrade to 24.1.2 appears to be downloading a massive base-24.1.2.txz

« on: February 20, 2024, 04:44:10 pm »

Just making sure things are going as expected...

Earlier this showed the expected changes:

Code: [Select]

Checking integrity... done (1 conflicting)
  - suricata-7.0.3 conflicts with suricata-stable-6.0.15 on /usr/local/bin/suricata
Checking integrity... done (0 conflicting)
Conflicts with the existing packages have been found.
One more solver iteration is needed to resolve them.
The following 25 package(s) will be affected (of 0 checked):

Installed packages to be REMOVED:
suricata-stable: 6.0.15

New packages to be INSTALLED:
suricata: 7.0.3

Installed packages to be UPGRADED:
boost-libs: 1.83.0_1 -> 1.84.0
clamav: 1.2.1_2,1 -> 1.3.0,1
crowdsec: 1.6.0 -> 1.6.0_1
crowdsec-firewall-bouncer: 0.0.28_2 -> 0.0.28_3
dnscrypt-proxy2: 2.0.45 -> 2.1.5_3
dnsmasq: 2.89_1,1 -> 2.90,1
kea: 2.4.1 -> 2.4.1_1
libidn2: 2.3.4_2 -> 2.3.7
libnghttp2: 1.58.0 -> 1.59.0
libucl: 0.8.2_1 -> 0.9.0
openvpn: 2.6.8_2 -> 2.6.9
opnsense: 24.1.1 -> 24.1.2
opnsense-update: 24.1 -> 24.1.2
os-dnscrypt-proxy: 1.14_1 -> 1.15
os-intrusion-detection-content-et-open: 1.0.2_1 -> 1.0.2_2
php82-phalcon: 5.3.1 -> 5.6.1
py39-pytz: 2023.3,1 -> 2024.1,1
py39-tzdata: 2023.4 -> 2024.1
radvd: 2.19_2 -> 2.19_3
sudo: 1.9.15p5_3 -> 1.9.15p5_4
telegraf: 1.29.4 -> 1.29.4_1
unbound: 1.19.0_1 -> 1.19.1

Installed packages to be REINSTALLED:
pkg-1.19.2_1

Number of packages to be removed: 1
Number of packages to be installed: 1
Number of packages to be upgraded: 22
Number of packages to be reinstalled: 1
How big is this / how long does this step take?

Code: [Select]

Fetching base-24.1.2-amd64.txz: ..............................................
It has been about 30 minutes or so on a 1 GB Fiber connection and the dots/pips keep showing up lol, just want to know I'm not caught in a loop, again, thank you for making such a powerful and useful tool/product!!

11

23.7 Legacy Series / Re: DHCP Dynamic DNS update to external BIND 9.18

« on: February 11, 2024, 03:44:51 am »

IMHO - It is far better to use /24s than attempt to subdivide your Homelab into subnets smaller than a /24, those Reverse DNS extra configs will be tedious and unless you have to have it, I would guide against that pattern.

Excellent reference into how one does it though! Not for the faint of heart lol.

12

24.1 Legacy Series / Re: CrowdSec 1.6.0 has been released, with 24.1 it appears we are still at 1.5.5

« on: February 11, 2024, 03:41:47 am »

Just saw it show up!! <3

13

24.1 Legacy Series / Re: Suricata 7 instead of Suricata 6 and no af-packet support?

« on: February 10, 2024, 08:46:21 pm »

> Overall curious about the decision and if OPNSense is compiling their own Suricata

I'm rather curious what the actual question is here. Someone has to provide binary packages so it needs to be built? oO
...

After having made my initial post there, I realized I have more to learn/explore before I can ask such a question really. While I have compiled my own Suricata on Debian, I have not on FreeBSD.

I'm after customizing the downloaded rules in greater context, it will still be SID by SID, but, ideally using Suicata-Update or something like the old "pulledpork" or "oinkmaster" that Snort had we would be able to have a conf file that SID by SID has modifications to apply after getting the new rule file (post update, it re-set the changes if the source detail to fix is still there).

How can I post rule update have specific rules modified (sed/awk - ish but through a pattern tool like Suricata-Update (gotta explore this - still learning) or something like it) "$ANY" to "$EXTERNAL" or "$INTERNAL"?

So, in addition to being curious about PF_RING or AF-PACKET, because memory rings tend to run fastest, and wanting to tune impact to an aging process IDS, wanted to go after that tuning of the IDS and the rules being enabled. Use to manage Snort IDS hosts for an MSP of sorts in the past, day job is a little different now, but my hobby and passion remain.

In a effort to gain awareness of my own network, current state of rules, I went after enabling almost all rules recently and wrote up a guide to getting a great many rules but turning off some of the most noisy/troublesome.
https://www.nova-labs.net/opnsense-and-enabling-suricata-rules/

I plan on posting about my future Suritcata-Update explorations in about another 3 to 4 weeks or so (I hope lol!)

14

24.1 Legacy Series / Re: Dynamic DNS for PTR Records with External BIND Server

« on: February 10, 2024, 08:15:13 pm »

As somebody that also does this, I've noticed it is strongly advised to use /24 subnets, reverse DNS w/less than a /24 requires very very custom/unique reverse DNS stanzas.

Outside of this... I've made sure to use "$ORIGIN" more and my external BIND doesn't seem to favor it, as well as making sure I only put "PTR" not "PTR IN" for the reverse, again BIND seems to favor the PTR IN, and OPNSense DHCP seems to favor the PTR.

You are doing AD/DC BIND? Not doing this myself, so, not sure how that would change the 'dance' if you will. Best of luck!!

15

Intrusion Detection and Prevention / How to enable via Policy and Rules useful Suricata IDS Rules (SIDs)

« on: February 02, 2024, 06:40:54 pm »

Looking to enable additional Suricata IDS Rules / SIDs? Just wrote a how-to w/screenshots, here we go!

TLDR;
https://www.nova-labs.net/opnsense-and-enabling-suricata-rules/

The how-to is a bit long, but outlined are three policy rules that once enabled allow a much wider/deeper view of the network traffic being inspected.

This will raise your CPU utilization, and if you do not add the third Policy, and disable a select few SIDs, can cause quite a bit of event/alert explosion as a few of the DNS/TLS/SNI rules fire each DNS resolution/TLS connection.

The guide starts by broadly enabling (first 2 policies), and then disabling (third policy) whole matching groups of rules based on the SID/rule meta. Thank you OPNSense, realized the population of each meta and then was able to focus on what to use to enable with minimal Policies.

Last section in the guide is where you will be individually disabling 20+ rules/SIDs which should not negatively impact your OPNSense router, we are keeping the individual rule mods in low populations.

Here's a first step before you even read the whole guide (you will likely want to have your OPNSense with a working internet connection to get through this guide and be able to get this initial step out of the way):


Please feel free to suggest modifications, or share your experience here.
Looking to learn more, but share what's being explored!