Messages

I'd like to see the 'Mode' (None/Active/Backup/Disabled) of a server in the Real Servers overview. Added this to github.

Immediately after upgrading to 23.10 (business edition) I clicked on check for updates again. This resulted:

Code Select Expand

***GOT REQUEST TO CHECK FOR UPDATES***
Currently running OPNsense 23.10 at Fri Jan  5 17:47:48 CET 2024
Fetching subscription information, please wait... done
Fetching changelog information, please wait... done
Updating OPNsense repository catalogue...
pkg: Repository OPNsense has a wrong packagesite, need to re-create database
Fetching meta.conf: . done
Fetching packagesite.pkg: .......... done
pkg: sqlite error while executing CREATE TABLE packages (id INTEGER PRIMARY KEY,origin TEXT,name TEXT NOT NULL,version TEXT NOT NULL,comment TEXT NOT NULL,desc TEXT NOT NULL,osversion TEXT,arch TEXT NOT NULL,maintainer TEXT NOT NULL,www TEXT,prefix TEXT NOT NULL,pkgsize INTEGER NOT NULL,flatsize INTEGER NOT NULL,licenselogic INTEGER NOT NULL,cksum TEXT NOT NULL,path TEXT NOT NULL,pkg_format_version INTEGER,manifestdigest TEXT NULL,olddigest TEXT NULL,dep_formula TEXT NULL,vital INTEGER NOT NULL DEFAULT 0);CREATE TABLE deps (origin TEXT,name TEXT,version TEXT,package_id INTEGER REFERENCES packages(id)  ON DELETE CASCADE ON UPDATE CASCADE,UNIQUE(package_id, name));CREATE TABLE categories (id INTEGER PRIMARY KEY, name TEXT NOT NULL UNIQUE );CREATE TABLE pkg_categories (package_id INTEGER REFERENCES packages(id)  ON DELETE CASCADE ON UPDATE CASCADE,category_id INTEGER REFERENCES categories(id)  ON DELETE RESTRICT ON UPDATE RESTRICT,UNIQUE(package_id, category_id));CREATE TABLE licenses (id INTEGER PRIMARY KEY,name TEXT NOT NULL UNIQUE);CREATE TABLE pkg_licenses (package_id INTEGER REFERENCES packages(id)  ON DELETE CASCADE ON UPDATE CASCADE,license_id INTEGER REFERENCES licenses(id)  ON DELETE RESTRICT ON UPDATE RESTRICT,UNIQUE(package_id, license_id));CREATE TABLE option (option_id INTEGER PRIMARY KEY,option TEXT NOT NULL UNIQUE);CREATE TABLE option_desc (option_desc_id INTEGER PRIMARY KEY,option_desc TEXT NOT NULL UNIQUE);CREATE TABLE pkg_option (package_id INTEGER NOT NULL REFERENCES packages(id) ON DELETE CASCADE ON UPDATE CASCADE,option_id INTEGER NOT NULL REFERENCES option(option_id) ON DELETE RESTRICT ON UPDATE CASCADE,value TEXT NOT NULL,PRIMARY KEY(package_id, option_id));CREATE TABLE pkg_option_desc (package_id INTEGER NOT NULL REFERENCES packages(id) ON DELETE CASCADE ON UPDATE CASCADE,option_id INTEGER NOT NULL REFERENCES option(option_id) ON DELETE RESTRICT ON UPDATE CASCADE,option_desc_id INTEGER NOT NULL REFERENCES option_desc(option_desc_id) ON DELETE RESTRICT ON UPDATE CASCADE,PRIMARY KEY(package_id, option_id));CREATE TABLE pkg_option_default (package_id INTEGER NOT NULL REFERENCES packages(id) ON DELETE CASCADE ON UPDATE CASCADE,option_id INTEGER NOT NULL REFERENCES option(option_id) ON DELETE RESTRICT ON UPDATE CASCADE,default_value TEXT NOT NULL,PRIMARY KEY(package_id, option_id));CREATE TABLE shlibs (id INTEGER PRIMARY KEY,name TEXT NOT NULL UNIQUE );CREATE TABLE pkg_shlibs_required (package_id INTEGER NOT NULL REFERENCES packages(id)  ON DELETE CASCADE ON UPDATE CASCADE,shlib_id INTEGER NOT NULL REFERENCES shlibs(id)  ON DELETE RESTRICT ON UPDATE RESTRICT,UNIQUE(package_id, shlib_id));CREATE TABLE pkg_shlibs_provided (package_id INTEGER NOT NULL REFERENCES packages(id)  ON DELETE CASCADE ON UPDATE CASCADE,shlib_id INTEGER NOT NULL REFERENCES shlibs(id)  ON DELETE RESTRICT ON UPDATE RESTRICT,UNIQUE(package_id, shlib_id));CREATE TABLE annotation (annotation_id INTEGER PRIMARY KEY,annotation TEXT NOT NULL UNIQUE);CREATE TABLE pkg_annotation (package_id INTEGER REFERENCES packages(id) ON DELETE CASCADE ON UPDATE RESTRICT,tag_id INTEGER NOT NULL REFERENCES annotation(annotation_id) ON DELETE CASCADE ON UPDATE RESTRICT,value_id INTEGER NOT NULL REFERENCES annotation(annotation_id) ON DELETE CASCADE ON UPDATE RESTRICT,UNIQUE (package_id, tag_id));CREATE TABLE pkg_conflicts (package_id INTEGER NOT NULL REFERENCES packages(id)  ON DELETE CASCADE ON UPDATE CASCADE,conflict_id INTEGER NOT NULL,UNIQUE(package_id, conflict_id));CREATE TABLE provides(    id INTEGER PRIMARY KEY,    provide TEXT NOT NULL);CREATE TABLE pkg_provides (package_id INTEGER NOT NULL REFERENCES packages(id)  ON DELETE CASCADE ON UPDATE CASCADE,provide_id INTEGER NOT NULL REFERENCES provides(id)  ON DELETE RESTRICT ON UPDATE RESTRICT,UNIQUE(package_id, provide_id));CREATE TABLE requires(    id INTEGER PRIMARY KEY,    require TEXT NOT NULL);CREATE TABLE pkg_requires (package_id INTEGER NOT NULL REFERENCES packages(id)  ON DELETE CASCADE ON UPDATE CASCADE,require_id INTEGER NOT NULL REFERENCES requires(id)  ON DELETE RESTRICT ON UPDATE RESTRICT,UNIQUE(package_id, require_id));PRAGMA user_version=2014; in file pkgdb.c:2333: attempt to write a readonly database
Unable to create repository OPNsense
Unable to update repository OPNsense
Error updating repositories!
pkg: Repository OPNsense cannot be opened. 'pkg update' required
Checking integrity... done (0 conflicting)
Your packages are up to date.
***DONE***

Harmless?

I got ET Open working. I have returned to ET Telemetry and everything is now working. A summary of the things that confused me:

• ET Telemetry comes with empty rule sets for botcc, dhsield, drop, ciarmy, compromised. You need to add an additional ET Open set (specifically for ET Telemetry) in System/Firmware/Plugins. My guess is there are SID conflicts between ET Telemetry and ET Open and hence you need an 'ET Telemetry safe' ET Open set, but that is a guess.
• When using ET Telemetry there is a 'Save' button and a 'Download & Update' button. If you change the enabled set, pressing Save gives a short living warning about having to Download & Update. Normally in UX (also in most parts of OPNsense, 'Save' is the final action if you change something, but here it is not the case. Also, a warning on Save generally means in UX that the Save has not proceeded. In combination with the previous point (empty sets) this gave the impression that 'Save' did not work. The UX here is: You Save your selection and with Download & Update you 'execute' that selection (which is a real 'save' too as this is when the rules actually change). So, I tried to get to a point where 'Save' would not give the warning, but it constantly did and Suricata also warned 'there were no rules'.
  (Note: ET Open does not have that 'Save' button. Probably a good solution would be to use 'Set Token' for the token and 'Apply' for the rest (as it is elsewhere in the UI))
  
• 
  A Policy is needed to change the default 'Alert' that is in almost all downloaded rulesets to 'Block'. Two things are confusing here. In Settings, when you turn on IPS, you may expect to go from 'inform me' to 'really block'.
• Trying the aubse.ch ThreatFox ruleset doesn't work together with setting a Policy. The Policy 'Apply' button gets a 'working' state and never finishes, not even after many hours.

So, the three things mixed here: ET Telemetry comes with a bunch of empty rule sets, turning IPS on doesn't block anything until you change 'alert' to 'block' in a Policy/Policies, and 'Save' is not (as it n romally is' the definitive action to get something working.

This happened when I load abuse.ch ThreatFox. It does not happen with my current set which excludes ThreatFox

One thing I ran into that confused me is that the policy has a 'from' action (at the top) and a 'to' action (at the end). The top action is by default set to 'Drop'. So, if you change the bottom one to 'Drop' you are in effect saying: if the action is 'Drop' change it to 'Drop'.

Make sure that you create a policy to change (Action, top) 'Alert' to (New Action, bottom)'Drop'.

To make sure the rules I have selected actually drop the traffic, I need to create a Policy that actually changes the default 'alert' on those rules to 'drop'.

So, I created such a rule, but when I clicked 'Apply' for the first time, it was not done after 30 minutes.

CPU usage is low, so what is it doing? Memory usage is high.

This one (a big one on almost all my rulesets) never finished:

Code Select Expand

# cat/usr/local/etc/suricata/rule-policies.config
[843a267bc7314362b09a08d4a25a9f51]
enabled=1
prio=0
rulesets=abuse.ch.feodotracker.rules,abuse.ch.sslblacklist.rules,abuse.ch.sslipblacklist.rules,abuse.ch.threatfox.rules,abuse.ch.urlhaus.rules,botcc.rules,ciarmy.rules,compromised.rules,drop.rules,dshield.rules,emerging-malware.rules,emerging-mobile_malware.rules,emerging-phishing.rules,emerging-web_client.rules,emerging-web_server.rules,opnsense.test.rules
content=
action=drop
__target_action__=drop
__policy_id__=843a267b-c731-4362-b09a-08d4a25a9f51
__policy_description__=Drop everything on these sets

What should I do to get my rulesets to actually block instead of just alert?

The answer is: you need to set a Policy.

(Not that it works yet, 'apply' never completes, but that is another issue. In theory it works.)

The Save button does not disappear. What happened is that it was available but when you clicked it it said "Download and Update" first (short time visible message). But if you then Download and Update, this stays.

The result was that I had no rules whatsoever.

But I am starting to suspect that I am misinterpreting the GUI. Save = 'save config", the Download & Update is the next step (to populate). Intuitively (or my intuition at least), the Save button would come at the end (which it normally does). As I did not see any actual rules (and Suricata complained), I concluded Save was failing.

Tomorrow I'm going to retry with ET Telemetry again. ET Open is working, but that one doesn't have a 'Save' button, only the "Download & Update".

I've moved from ET Telemetry Pro to ET Open and I have activated a set of rules.

I now see Alerts in IDS/IPS like this:

Code Select Expand

ET COMPROMISED Known Compromised or Hostile Host Traffic group 18 but the action is 'allowed'.

Why?

Remove ET Telemetry Pro and installed ET Open. That one works. The frontend is also different, there is no Save button (for now, who knows what happens when OPNsense gets hosed again here too). Simply update the rules.

[removed as potentially wrong]

Forget it. I have deinstalled and reinstalled the ET Telemetry version and I no longer have this error, but still the frontend Save button for Downloads refuses to work.

Kind of pissed now about losing 7 hours with an OPNsense frontend that suddenly for some unknown reason when I was changing the IDS config has stopped working. I'd like a 'total reset' option for my suricata config. CLI is fine.

It seems OPNsense can get i a state where its frontend UI stops working and als stops creating a usable Suricata config because the 'Save' button won't work. And it seems that state for the UI is permanent and survives whatever you install/deinstall as plugins.

[Removed, it was a red herring]

I keep getting this when trying to save my Suricata download set in Administration. Even deselecting everything and trying to save gets me this. Download & Update rules doesn't help.

I can start suricata, but it says 'no rules are loaded' so it is now completely nonfunctional.

Help?

OPNsense 22.10.2 (Deciso)

Log shows error:

Code Select Expand

2023-04-21T14:42:01 Warning suricata [100410] <Warning> -- [ERRCODE: SC_ERR_NO_RULES_LOADED(43)] - 2 rule files specified, but no rules were loaded!

This is very simple. Make sure that the OPNsense test rules package is installed: "OPNsense-App-detect/test".
Then you can download e.g. the Eicar testvirus via http:

"http://www.eicar.org/download/eicar.com"
If you then check your alerts, you should find a blocking event and not be able to download the file. Your browser or curl will then run into a timeout.

Thank you and sorry for the late thank you. I actually forgot I already asked and I was distracted.

I recently asked again because I found https://secure.eicar.org/eicar.com and I could download this one. But then I thought, moment, that is inside https so Suricata will not be able to see it and then I thought "given that all that web traffic is inside SSL, what use if Suricata for web traffic?". But that is more a suricata forum question