Topics

1

Zenarmor (Sensei) / MongoDB database locked

« on: August 26, 2023, 09:57:32 pm »

A few moments ago my firewall locked up and we lost internet and local network access.

I was able to reboot through the console and it looks like we are back up.

Looking at the logs I see the following warnings:

Code: [Select]

Date
Severity
Process
Line
2023-08-26T12:02:44-07:00 Error configd.py Timeout (120) executing : zenarmor database-disk-size /usr/local/datastore/mongodb
2023-08-26T11:57:18-07:00 Error configd.py Timeout (120) executing : zenarmor service mongod status
2023-08-26T11:50:25-07:00 Error configd.py Timeout (120) executing : zenarmor service mongod status
2023-08-26T11:02:11-07:00 Error configd.py [698dfda4-490d-4e98-8266-618ad74e3b65] Script action stderr returned "b'Parse error near line 1: database is locked (5)\nParse error near line 1: database is locked (5)\nParse error near line 1: database is locked (5)\nParse error near line 1: database is locked (5)\nParse error near line 1: database is locked (5)\nParse error nea'"
... plus a number of subsequent timeout errors of the same type.

Is any action recommended other than the occasional reboot? I'm using native netmap drivers.


Type   opnsense   
Version   23.7.1_3   
Architecture   amd64   
Commit   239f8d1f8   
Mirror   https://pkg.opnsense.org/FreeBSD:13:amd64/23.7   
Repositories   OPNsense, SunnyValley   
Updated on   Thu Aug 17 09:32:10 MST 2023

ZenArmor:
Engine
Status:   Running
Version:   1.14.4 - Aug 23, 2023 10:12 AM

Disk space
Usage:   9.0 GB / 421.9 GB
Mount:   /

Usage:   599.3 MB / 413.4 GB
Mount:   /var/log

etc.

2

23.7 Legacy Series / Update check fails, sqlite error

« on: August 11, 2023, 04:39:27 pm »

Hello,

I'm in the process of reconfiguring firewall security after multiple issues/crashes with ZenArmor.

I have done a clean re-installation of OPNsense and configured AdGuard Home and Crowdsec. After testing, I added Suricata with a couple of rules.

This morning I checked for updates and received an error. I then disabled Suricata, rebooted, and re-tried. I received same the error message below. I observed similar messages when I was having trouble with ZenArmor.

Hardware: Protectli VP 2410 16 GB, 500 GB SSD
OPNsense 23.7.1_3 (updated yesterday so I think I'm up-to-date)
AdGuard Home, Crowdsec, and Smart add-ons

Memory, Disk, and CPU usage are all very low according to the Dashboard. The UI is very responsive. Smartctl reports PASSED (no errors). I've also done MEMtest.

How do I resolve this?

Code: [Select]

***GOT REQUEST TO CHECK FOR UPDATES***
Currently running OPNsense 23.7.1_3 at Fri Aug 11 07:15:09 MST 2023
Fetching changelog information, please wait... done
Updating OPNsense repository catalogue...
pkg: sqlite error while executing SELECT count(name) FROM sqlite_master WHERE type='table' AND name='repodata'; in file pkgdb.c:2358: database is locked
pkg: Repository OPNsense contains no repodata table, need to re-create database
Fetching meta.conf: . done
Fetching packagesite.pkg: .......... done
pkg: sqlite error while executing CREATE TABLE packages (id INTEGER PRIMARY KEY,origin TEXT,name TEXT NOT NULL,version TEXT NOT NULL,comment TEXT NOT NULL,desc TEXT NOT NULL,osversion TEXT,arch TEXT NOT NULL,maintainer TEXT NOT NULL,www TEXT,prefix TEXT NOT NULL,pkgsize INTEGER NOT NULL,flatsize INTEGER NOT NULL,licenselogic INTEGER NOT NULL,cksum TEXT NOT NULL,path TEXT NOT NULL,pkg_format_version INTEGER,manifestdigest TEXT NULL,olddigest TEXT NULL,dep_formula TEXT NULL,vital INTEGER NOT NULL DEFAULT 0);CREATE TABLE deps (origin TEXT,name TEXT,version TEXT,package_id INTEGER REFERENCES packages(id)  ON DELETE CASCADE ON UPDATE CASCADE,UNIQUE(package_id, name));CREATE TABLE categories (id INTEGER PRIMARY KEY, name TEXT NOT NULL UNIQUE );CREATE TABLE pkg_categories (package_id INTEGER REFERENCES packages(id)  ON DELETE CASCADE ON UPDATE CASCADE,category_id INTEGER REFERENCES categories(id)  ON DELETE RESTRICT ON UPDATE RESTRICT,UNIQUE(package_id, category_id));CREATE TABLE licenses (id INTEGER PRIMARY KEY,name TEXT NOT NULL UNIQUE);CREATE TABLE pkg_licenses (package_id INTEGER REFERENCES packages(id)  ON DELETE CASCADE ON UPDATE CASCADE,license_id INTEGER REFERENCES licenses(id)  ON DELETE RESTRICT ON UPDATE RESTRICT,UNIQUE(package_id, license_id));CREATE TABLE option (option_id INTEGER PRIMARY KEY,option TEXT NOT NULL UNIQUE);CREATE TABLE option_desc (option_desc_id INTEGER PRIMARY KEY,option_desc TEXT NOT NULL UNIQUE);CREATE TABLE pkg_option (package_id INTEGER NOT NULL REFERENCES packages(id) ON DELETE CASCADE ON UPDATE CASCADE,option_id INTEGER NOT NULL REFERENCES option(option_id) ON DELETE RESTRICT ON UPDATE CASCADE,value TEXT NOT NULL,PRIMARY KEY(package_id, option_id));CREATE TABLE pkg_option_desc (package_id INTEGER NOT NULL REFERENCES packages(id) ON DELETE CASCADE ON UPDATE CASCADE,option_id INTEGER NOT NULL REFERENCES option(option_id) ON DELETE RESTRICT ON UPDATE CASCADE,option_desc_id INTEGER NOT NULL REFERENCES option_desc(option_desc_id) ON DELETE RESTRICT ON UPDATE CASCADE,PRIMARY KEY(package_id, option_id));CREATE TABLE pkg_option_default (package_id INTEGER NOT NULL REFERENCES packages(id) ON DELETE CASCADE ON UPDATE CASCADE,option_id INTEGER NOT NULL REFERENCES option(option_id) ON DELETE RESTRICT ON UPDATE CASCADE,default_value TEXT NOT NULL,PRIMARY KEY(package_id, option_id));CREATE TABLE shlibs (id INTEGER PRIMARY KEY,name TEXT NOT NULL UNIQUE );CREATE TABLE pkg_shlibs_required (package_id INTEGER NOT NULL REFERENCES packages(id)  ON DELETE CASCADE ON UPDATE CASCADE,shlib_id INTEGER NOT NULL REFERENCES shlibs(id)  ON DELETE RESTRICT ON UPDATE RESTRICT,UNIQUE(package_id, shlib_id));CREATE TABLE pkg_shlibs_provided (package_id INTEGER NOT NULL REFERENCES packages(id)  ON DELETE CASCADE ON UPDATE CASCADE,shlib_id INTEGER NOT NULL REFERENCES shlibs(id)  ON DELETE RESTRICT ON UPDATE RESTRICT,UNIQUE(package_id, shlib_id));CREATE TABLE annotation (annotation_id INTEGER PRIMARY KEY,annotation TEXT NOT NULL UNIQUE);CREATE TABLE pkg_annotation (package_id INTEGER REFERENCES packages(id) ON DELETE CASCADE ON UPDATE RESTRICT,tag_id INTEGER NOT NULL REFERENCES annotation(annotation_id) ON DELETE CASCADE ON UPDATE RESTRICT,value_id INTEGER NOT NULL REFERENCES annotation(annotation_id) ON DELETE CASCADE ON UPDATE RESTRICT,UNIQUE (package_id, tag_id));CREATE TABLE pkg_conflicts (package_id INTEGER NOT NULL REFERENCES packages(id)  ON DELETE CASCADE ON UPDATE CASCADE,conflict_id INTEGER NOT NULL,UNIQUE(package_id, conflict_id));CREATE TABLE provides(    id INTEGER PRIMARY KEY,    provide TEXT NOT NULL);CREATE TABLE pkg_provides (package_id INTEGER NOT NULL REFERENCES packages(id)  ON DELETE CASCADE ON UPDATE CASCADE,provide_id INTEGER NOT NULL REFERENCES provides(id)  ON DELETE RESTRICT ON UPDATE RESTRICT,UNIQUE(package_id, provide_id));CREATE TABLE requires(    id INTEGER PRIMARY KEY,    require TEXT NOT NULL);CREATE TABLE pkg_requires (package_id INTEGER NOT NULL REFERENCES packages(id)  ON DELETE CASCADE ON UPDATE CASCADE,require_id INTEGER NOT NULL REFERENCES requires(id)  ON DELETE RESTRICT ON UPDATE RESTRICT,UNIQUE(package_id, require_id));PRAGMA user_version=2014; in file pkgdb.c:2333: disk I/O error
Unable to create repository OPNsense
Unable to update repository OPNsense
Updating mimugmail repository catalogue...
Fetching meta.conf: . done
Fetching packagesite.pkg: ........ done
pkg: sqlite error while executing CREATE TABLE packages (id INTEGER PRIMARY KEY,origin TEXT,name TEXT NOT NULL,version TEXT NOT NULL,comment TEXT NOT NULL,desc TEXT NOT NULL,osversion TEXT,arch TEXT NOT NULL,maintainer TEXT NOT NULL,www TEXT,prefix TEXT NOT NULL,pkgsize INTEGER NOT NULL,flatsize INTEGER NOT NULL,licenselogic INTEGER NOT NULL,cksum TEXT NOT NULL,path TEXT NOT NULL,pkg_format_version INTEGER,manifestdigest TEXT NULL,olddigest TEXT NULL,dep_formula TEXT NULL,vital INTEGER NOT NULL DEFAULT 0);CREATE TABLE deps (origin TEXT,name TEXT,version TEXT,package_id INTEGER REFERENCES packages(id)  ON DELETE CASCADE ON UPDATE CASCADE,UNIQUE(package_id, name));CREATE TABLE categories (id INTEGER PRIMARY KEY, name TEXT NOT NULL UNIQUE );CREATE TABLE pkg_categories (package_id INTEGER REFERENCES packages(id)  ON DELETE CASCADE ON UPDATE CASCADE,category_id INTEGER REFERENCES categories(id)  ON DELETE RESTRICT ON UPDATE RESTRICT,UNIQUE(package_id, category_id));CREATE TABLE licenses (id INTEGER PRIMARY KEY,name TEXT NOT NULL UNIQUE);CREATE TABLE pkg_licenses (package_id INTEGER REFERENCES packages(id)  ON DELETE CASCADE ON UPDATE CASCADE,license_id INTEGER REFERENCES licenses(id)  ON DELETE RESTRICT ON UPDATE RESTRICT,UNIQUE(package_id, license_id));CREATE TABLE option (option_id INTEGER PRIMARY KEY,option TEXT NOT NULL UNIQUE);CREATE TABLE option_desc (option_desc_id INTEGER PRIMARY KEY,option_desc TEXT NOT NULL UNIQUE);CREATE TABLE pkg_option (package_id INTEGER NOT NULL REFERENCES packages(id) ON DELETE CASCADE ON UPDATE CASCADE,option_id INTEGER NOT NULL REFERENCES option(option_id) ON DELETE RESTRICT ON UPDATE CASCADE,value TEXT NOT NULL,PRIMARY KEY(package_id, option_id));CREATE TABLE pkg_option_desc (package_id INTEGER NOT NULL REFERENCES packages(id) ON DELETE CASCADE ON UPDATE CASCADE,option_id INTEGER NOT NULL REFERENCES option(option_id) ON DELETE RESTRICT ON UPDATE CASCADE,option_desc_id INTEGER NOT NULL REFERENCES option_desc(option_desc_id) ON DELETE RESTRICT ON UPDATE CASCADE,PRIMARY KEY(package_id, option_id));CREATE TABLE pkg_option_default (package_id INTEGER NOT NULL REFERENCES packages(id) ON DELETE CASCADE ON UPDATE CASCADE,option_id INTEGER NOT NULL REFERENCES option(option_id) ON DELETE RESTRICT ON UPDATE CASCADE,default_value TEXT NOT NULL,PRIMARY KEY(package_id, option_id));CREATE TABLE shlibs (id INTEGER PRIMARY KEY,name TEXT NOT NULL UNIQUE );CREATE TABLE pkg_shlibs_required (package_id INTEGER NOT NULL REFERENCES packages(id)  ON DELETE CASCADE ON UPDATE CASCADE,shlib_id INTEGER NOT NULL REFERENCES shlibs(id)  ON DELETE RESTRICT ON UPDATE RESTRICT,UNIQUE(package_id, shlib_id));CREATE TABLE pkg_shlibs_provided (package_id INTEGER NOT NULL REFERENCES packages(id)  ON DELETE CASCADE ON UPDATE CASCADE,shlib_id INTEGER NOT NULL REFERENCES shlibs(id)  ON DELETE RESTRICT ON UPDATE RESTRICT,UNIQUE(package_id, shlib_id));CREATE TABLE annotation (annotation_id INTEGER PRIMARY KEY,annotation TEXT NOT NULL UNIQUE);CREATE TABLE pkg_annotation (package_id INTEGER REFERENCES packages(id) ON DELETE CASCADE ON UPDATE RESTRICT,tag_id INTEGER NOT NULL REFERENCES annotation(annotation_id) ON DELETE CASCADE ON UPDATE RESTRICT,value_id INTEGER NOT NULL REFERENCES annotation(annotation_id) ON DELETE CASCADE ON UPDATE RESTRICT,UNIQUE (package_id, tag_id));CREATE TABLE pkg_conflicts (package_id INTEGER NOT NULL REFERENCES packages(id)  ON DELETE CASCADE ON UPDATE CASCADE,conflict_id INTEGER NOT NULL,UNIQUE(package_id, conflict_id));CREATE TABLE provides(    id INTEGER PRIMARY KEY,    provide TEXT NOT NULL);CREATE TABLE pkg_provides (package_id INTEGER NOT NULL REFERENCES packages(id)  ON DELETE CASCADE ON UPDATE CASCADE,provide_id INTEGER NOT NULL REFERENCES provides(id)  ON DELETE RESTRICT ON UPDATE RESTRICT,UNIQUE(package_id, provide_id));CREATE TABLE requires(    id INTEGER PRIMARY KEY,    require TEXT NOT NULL);CREATE TABLE pkg_requires (package_id INTEGER NOT NULL REFERENCES packages(id)  ON DELETE CASCADE ON UPDATE CASCADE,require_id INTEGER NOT NULL REFERENCES requires(id)  ON DELETE RESTRICT ON UPDATE RESTRICT,UNIQUE(package_id, require_id));PRAGMA user_version=2014; in file pkgdb.c:2333: attempt to write a readonly database
Unable to create repository mimugmail
Unable to update repository mimugmail
Error updating repositories!
pkg: Repository OPNsense cannot be opened. 'pkg update' required
Checking integrity... done (0 conflicting)
Your packages are up to date.
***DONE***

3

Zenarmor (Sensei) / RESOLVED: ZenArmor UI not loading, update failure

« on: August 08, 2023, 05:46:00 pm »

Yesterday I wiped my firewall and loaded OPNsense 23.7 and added Zenarmor 1.14 using MongoDB.
https://forum.opnsense.org/index.php?topic=35012.0

All seemed well at first, but now the UI will not load, and OPNsense update fails due (apparently) to a database issue. I'm posting here because this seems to be related to other ZA issues I have experienced recently. FYI I've seen a "Database is locked" message recently when trying to view the ZA dashboard.

From what I can tell ZA seems to be running.

Should I remove ZA and re-test?

Code: [Select]

***GOT REQUEST TO CHECK FOR UPDATES***
Currently running OPNsense 23.7 at Tue Aug  8 08:28:28 MST 2023
Fetching changelog information, please wait... done
Updating OPNsense repository catalogue...
Fetching meta.conf: . done
Fetching packagesite.pkg: .......... done
pkg: sqlite error while executing CREATE TABLE packages (id INTEGER PRIMARY KEY,origin TEXT,name TEXT NOT NULL,version TEXT NOT NULL,comment TEXT NOT NULL,desc TEXT NOT NULL,osversion TEXT,arch TEXT NOT NULL,maintainer TEXT NOT NULL,www TEXT,prefix TEXT NOT NULL,pkgsize INTEGER NOT NULL,flatsize INTEGER NOT NULL,licenselogic INTEGER NOT NULL,cksum TEXT NOT NULL,path TEXT NOT NULL,pkg_format_version INTEGER,manifestdigest TEXT NULL,olddigest TEXT NULL,dep_formula TEXT NULL,vital INTEGER NOT NULL DEFAULT 0);CREATE TABLE deps (origin TEXT,name TEXT,version TEXT,package_id INTEGER REFERENCES packages(id)  ON DELETE CASCADE ON UPDATE CASCADE,UNIQUE(package_id, name));CREATE TABLE categories (id INTEGER PRIMARY KEY, name TEXT NOT NULL UNIQUE );CREATE TABLE pkg_categories (package_id INTEGER REFERENCES packages(id)  ON DELETE CASCADE ON UPDATE CASCADE,category_id INTEGER REFERENCES categories(id)  ON DELETE RESTRICT ON UPDATE RESTRICT,UNIQUE(package_id, category_id));CREATE TABLE licenses (id INTEGER PRIMARY KEY,name TEXT NOT NULL UNIQUE);CREATE TABLE pkg_licenses (package_id INTEGER REFERENCES packages(id)  ON DELETE CASCADE ON UPDATE CASCADE,license_id INTEGER REFERENCES licenses(id)  ON DELETE RESTRICT ON UPDATE RESTRICT,UNIQUE(package_id, license_id));CREATE TABLE option (option_id INTEGER PRIMARY KEY,option TEXT NOT NULL UNIQUE);CREATE TABLE option_desc (option_desc_id INTEGER PRIMARY KEY,option_desc TEXT NOT NULL UNIQUE);CREATE TABLE pkg_option (package_id INTEGER NOT NULL REFERENCES packages(id) ON DELETE CASCADE ON UPDATE CASCADE,option_id INTEGER NOT NULL REFERENCES option(option_id) ON DELETE RESTRICT ON UPDATE CASCADE,value TEXT NOT NULL,PRIMARY KEY(package_id, option_id));CREATE TABLE pkg_option_desc (package_id INTEGER NOT NULL REFERENCES packages(id) ON DELETE CASCADE ON UPDATE CASCADE,option_id INTEGER NOT NULL REFERENCES option(option_id) ON DELETE RESTRICT ON UPDATE CASCADE,option_desc_id INTEGER NOT NULL REFERENCES option_desc(option_desc_id) ON DELETE RESTRICT ON UPDATE CASCADE,PRIMARY KEY(package_id, option_id));CREATE TABLE pkg_option_default (package_id INTEGER NOT NULL REFERENCES packages(id) ON DELETE CASCADE ON UPDATE CASCADE,option_id INTEGER NOT NULL REFERENCES option(option_id) ON DELETE RESTRICT ON UPDATE CASCADE,default_value TEXT NOT NULL,PRIMARY KEY(package_id, option_id));CREATE TABLE shlibs (id INTEGER PRIMARY KEY,name TEXT NOT NULL UNIQUE );CREATE TABLE pkg_shlibs_required (package_id INTEGER NOT NULL REFERENCES packages(id)  ON DELETE CASCADE ON UPDATE CASCADE,shlib_id INTEGER NOT NULL REFERENCES shlibs(id)  ON DELETE RESTRICT ON UPDATE RESTRICT,UNIQUE(package_id, shlib_id));CREATE TABLE pkg_shlibs_provided (package_id INTEGER NOT NULL REFERENCES packages(id)  ON DELETE CASCADE ON UPDATE CASCADE,shlib_id INTEGER NOT NULL REFERENCES shlibs(id)  ON DELETE RESTRICT ON UPDATE RESTRICT,UNIQUE(package_id, shlib_id));CREATE TABLE annotation (annotation_id INTEGER PRIMARY KEY,annotation TEXT NOT NULL UNIQUE);CREATE TABLE pkg_annotation (package_id INTEGER REFERENCES packages(id) ON DELETE CASCADE ON UPDATE RESTRICT,tag_id INTEGER NOT NULL REFERENCES annotation(annotation_id) ON DELETE CASCADE ON UPDATE RESTRICT,value_id INTEGER NOT NULL REFERENCES annotation(annotation_id) ON DELETE CASCADE ON UPDATE RESTRICT,UNIQUE (package_id, tag_id));CREATE TABLE pkg_conflicts (package_id INTEGER NOT NULL REFERENCES packages(id)  ON DELETE CASCADE ON UPDATE CASCADE,conflict_id INTEGER NOT NULL,UNIQUE(package_id, conflict_id));CREATE TABLE provides(    id INTEGER PRIMARY KEY,    provide TEXT NOT NULL);CREATE TABLE pkg_provides (package_id INTEGER NOT NULL REFERENCES packages(id)  ON DELETE CASCADE ON UPDATE CASCADE,provide_id INTEGER NOT NULL REFERENCES provides(id)  ON DELETE RESTRICT ON UPDATE RESTRICT,UNIQUE(package_id, provide_id));CREATE TABLE requires(    id INTEGER PRIMARY KEY,    require TEXT NOT NULL);CREATE TABLE pkg_requires (package_id INTEGER NOT NULL REFERENCES packages(id)  ON DELETE CASCADE ON UPDATE CASCADE,require_id INTEGER NOT NULL REFERENCES requires(id)  ON DELETE RESTRICT ON UPDATE RESTRICT,UNIQUE(package_id, require_id));PRAGMA user_version=2014; in file pkgdb.c:2333: attempt to write a readonly database
Unable to create repository OPNsense
Unable to update repository OPNsense
Updating SunnyValley repository catalogue...
Fetching meta.conf: . done
Fetching packagesite.pkg: .. done
pkg: sqlite error while executing CREATE TABLE packages (id INTEGER PRIMARY KEY,origin TEXT,name TEXT NOT NULL,version TEXT NOT NULL,comment TEXT NOT NULL,desc TEXT NOT NULL,osversion TEXT,arch TEXT NOT NULL,maintainer TEXT NOT NULL,www TEXT,prefix TEXT NOT NULL,pkgsize INTEGER NOT NULL,flatsize INTEGER NOT NULL,licenselogic INTEGER NOT NULL,cksum TEXT NOT NULL,path TEXT NOT NULL,pkg_format_version INTEGER,manifestdigest TEXT NULL,olddigest TEXT NULL,dep_formula TEXT NULL,vital INTEGER NOT NULL DEFAULT 0);CREATE TABLE deps (origin TEXT,name TEXT,version TEXT,package_id INTEGER REFERENCES packages(id)  ON DELETE CASCADE ON UPDATE CASCADE,UNIQUE(package_id, name));CREATE TABLE categories (id INTEGER PRIMARY KEY, name TEXT NOT NULL UNIQUE );CREATE TABLE pkg_categories (package_id INTEGER REFERENCES packages(id)  ON DELETE CASCADE ON UPDATE CASCADE,category_id INTEGER REFERENCES categories(id)  ON DELETE RESTRICT ON UPDATE RESTRICT,UNIQUE(package_id, category_id));CREATE TABLE licenses (id INTEGER PRIMARY KEY,name TEXT NOT NULL UNIQUE);CREATE TABLE pkg_licenses (package_id INTEGER REFERENCES packages(id)  ON DELETE CASCADE ON UPDATE CASCADE,license_id INTEGER REFERENCES licenses(id)  ON DELETE RESTRICT ON UPDATE RESTRICT,UNIQUE(package_id, license_id));CREATE TABLE option (option_id INTEGER PRIMARY KEY,option TEXT NOT NULL UNIQUE);CREATE TABLE option_desc (option_desc_id INTEGER PRIMARY KEY,option_desc TEXT NOT NULL UNIQUE);CREATE TABLE pkg_option (package_id INTEGER NOT NULL REFERENCES packages(id) ON DELETE CASCADE ON UPDATE CASCADE,option_id INTEGER NOT NULL REFERENCES option(option_id) ON DELETE RESTRICT ON UPDATE CASCADE,value TEXT NOT NULL,PRIMARY KEY(package_id, option_id));CREATE TABLE pkg_option_desc (package_id INTEGER NOT NULL REFERENCES packages(id) ON DELETE CASCADE ON UPDATE CASCADE,option_id INTEGER NOT NULL REFERENCES option(option_id) ON DELETE RESTRICT ON UPDATE CASCADE,option_desc_id INTEGER NOT NULL REFERENCES option_desc(option_desc_id) ON DELETE RESTRICT ON UPDATE CASCADE,PRIMARY KEY(package_id, option_id));CREATE TABLE pkg_option_default (package_id INTEGER NOT NULL REFERENCES packages(id) ON DELETE CASCADE ON UPDATE CASCADE,option_id INTEGER NOT NULL REFERENCES option(option_id) ON DELETE RESTRICT ON UPDATE CASCADE,default_value TEXT NOT NULL,PRIMARY KEY(package_id, option_id));CREATE TABLE shlibs (id INTEGER PRIMARY KEY,name TEXT NOT NULL UNIQUE );CREATE TABLE pkg_shlibs_required (package_id INTEGER NOT NULL REFERENCES packages(id)  ON DELETE CASCADE ON UPDATE CASCADE,shlib_id INTEGER NOT NULL REFERENCES shlibs(id)  ON DELETE RESTRICT ON UPDATE RESTRICT,UNIQUE(package_id, shlib_id));CREATE TABLE pkg_shlibs_provided (package_id INTEGER NOT NULL REFERENCES packages(id)  ON DELETE CASCADE ON UPDATE CASCADE,shlib_id INTEGER NOT NULL REFERENCES shlibs(id)  ON DELETE RESTRICT ON UPDATE RESTRICT,UNIQUE(package_id, shlib_id));CREATE TABLE annotation (annotation_id INTEGER PRIMARY KEY,annotation TEXT NOT NULL UNIQUE);CREATE TABLE pkg_annotation (package_id INTEGER REFERENCES packages(id) ON DELETE CASCADE ON UPDATE RESTRICT,tag_id INTEGER NOT NULL REFERENCES annotation(annotation_id) ON DELETE CASCADE ON UPDATE RESTRICT,value_id INTEGER NOT NULL REFERENCES annotation(annotation_id) ON DELETE CASCADE ON UPDATE RESTRICT,UNIQUE (package_id, tag_id));CREATE TABLE pkg_conflicts (package_id INTEGER NOT NULL REFERENCES packages(id)  ON DELETE CASCADE ON UPDATE CASCADE,conflict_id INTEGER NOT NULL,UNIQUE(package_id, conflict_id));CREATE TABLE provides(    id INTEGER PRIMARY KEY,    provide TEXT NOT NULL);CREATE TABLE pkg_provides (package_id INTEGER NOT NULL REFERENCES packages(id)  ON DELETE CASCADE ON UPDATE CASCADE,provide_id INTEGER NOT NULL REFERENCES provides(id)  ON DELETE RESTRICT ON UPDATE RESTRICT,UNIQUE(package_id, provide_id));CREATE TABLE requires(    id INTEGER PRIMARY KEY,    require TEXT NOT NULL);CREATE TABLE pkg_requires (package_id INTEGER NOT NULL REFERENCES packages(id)  ON DELETE CASCADE ON UPDATE CASCADE,require_id INTEGER NOT NULL REFERENCES requires(id)  ON DELETE RESTRICT ON UPDATE RESTRICT,UNIQUE(package_id, require_id));PRAGMA user_version=2014; in file pkgdb.c:2333: attempt to write a readonly database
Unable to create repository SunnyValley
Unable to update repository SunnyValley
Error updating repositories!
pkg: Repository OPNsense cannot be opened. 'pkg update' required
pkg: Cannot get an advisory lock on a database, it is locked by another process
***DONE***

4

Zenarmor (Sensei) / [Resolved]: Considering reinstallation of ZenArmor after major crash

« on: July 24, 2023, 04:15:47 pm »

I updated OPNsense late last week, and my system went down hard. Details pointed to an issue with netmap and ZenArmor.  See:
https://forum.opnsense.org/index.php?topic=34992.0

I'm running OPNsense on a Protectli VP2410, which has four Intel GB NIC's. Ports 0 and 1 are assigned to WAN and LAN, and 2-3 are set to LACP with a few VLANS.

I've been running Zenarmor for several months with no issue until this. So I assume that something "went wrong" with the update. Curiously, the errors I saw indicated "Emulated netmap" even though I had configured "Native netmap" drivers when I configured ZenArmor in the UI.

There were also "possible flapping" errors with my two LAGG ports.

I wish to reinstall ZenArmor, but I could use some guidance. Considering that I have two LAGG/LACP ports, how should I configure the netmap drivers for ZenArmor? Have all the issues with emulated netmap been addressed? I understand It was a big issue several months ago.

Any guidance would be appreciated.

5

23.1 Legacy Series / OPNsense down after update, multiple netmap errors, Zenarmor conflict?

« on: July 23, 2023, 12:33:24 am »

Hi all,

Yesterday I updated OPNsense to (what I presume is) the latest production version. It look a couple of reboots to bring the network back up. All seemed well for several hours, then the network went down hard. (Prior to this OPNsense has been running great as-configured below.)

Hardware: Protectli VP2410 (four port), 16 MB DRAM, 250 GB SSD

After restoring an old backup (see below):
OPNsense 23.1.11-amd64
FreeBSD 13.1-RELEASE-p8
OpenSSL 1.1.1u 30 May 2023

Assignments: Wan (igb0), LAN (igb1), LACP (igb 2 and 3) with a few vlans

Plugins: os-sunnyvalley, os-sensei, os-sensei-agent, os-sensei-updater
ZenArmor is configured with elasticsearch.

Today I reconnected the console and I’ve rebooted a few times. I’ve also restored backups from June via the console. My network does come up briefly after a ~25min reboot cycle (!), but goes back down (occasionally going up again for a few moments).

BTW my version after update showed 23.1.11, and I don’t think that’s the latest.

My console shows a large number of messages relating to netmap and emulated adapters. For example:

… generic_netmap_unregister Emulated adapter for vlan02 deactivated
…generic_netmap_d tor Emulated adapter for vlan02 destroyed

Same for vlans 03 and 04.

Watching the console for awhile I see:

Igb2 (and 3): Interface stopped DISTRIBUTING, possible flapping

I spent hours troubleshooting and rebooting etc. Checked and rebooted my switch. Access to the UI was hit and mostly-miss. I recall from some time ago that there could be some issues with LAGG and netmap with ZenArmor, so I put ZenArmor packet engine in bypass mode after several attempts. This seemed to help. But the same errors reappeared. Next I stopped all three ZenArmor services (packet engine, Elasticsearch, and Cloud agent) and set each NOT to start on reboot. Seeing the same behavior, I rebooted again.

Now OPNsense is running and my network is back up. I do not see netmap or other errors on the console. So my 'emergency' is contained. 

However, the UI is sluggish and I'm unable to update OPNsense from the UI.

pkg: sqlite error while executing CREATE TABLE packages ...

Could someone please offer some help/advice? I'd appreciate it!

Thanks, Doug

6

Zenarmor (Sensei) / Zenarmor installation crashes OPNsense. Reboot hangs.

« on: February 24, 2023, 04:25:27 am »

Hello community,

I'm a new user with OPNsense, running a Protectli VP2420 (J412 Celeron, 16 MB RAM, 500 GB SSD).

Installation of Sensei/Zenarmor crashes my Protectli running OPNsense. It's happened three times now, and each crash is hard (requires reinstalling OPNsense and reconfiguring). I'm running the Protectli as my home router, and the crashes are more than inconvenient (home automation down, lots of reconfiguring).

With the last two attempts I see the following on the OPNsense CLI when the boot proces hangs:
KDB: enter panic
[ thread PID 54 tid 1001106]
Stopped at kdb_enter+0x37: movq $0.0x11f86efe(%rip)
db>

This occurs with boot into "safe mode;" std. boot also hangs. It just takes longer.

Network configuration:
Modem -> Protectli OPNsense WAN -> LAN -> Switch 1 (PC 1 – monitors OPNsense etc., Eero wifi router – double-NAT, two-port LACP to router 2), -> Router 2 (PC2, two port LACP to Switch 1).

I suspected that LACP between my two switches was an issue, so I removed LACP and rebooted OPNsense and tried installing ZenArmor. Same crash and subsequent boot hang. (Note that OPNsense was NOT configured with LACP in any case.)

I'm a bit concerned about the double-NAT with the Eero router, but the darn thing doesn't seem to work in bridge mode in this configuration. Most of my devices are WiFi. 

Are there any Sensei's out there who can help me? I wish to run Zenarmor, and my router is supposed to be up-to-the-task.

Doug_Phoenix