RESOLVED: ZenArmor UI not loading, update failure

[doug_phoenix]

Yesterday I wiped my firewall and loaded OPNsense 23.7 and added Zenarmor 1.14 using MongoDB.
https://forum.opnsense.org/index.php?topic=35012.0

All seemed well at first, but now the UI will not load, and OPNsense update fails due (apparently) to a database issue. I'm posting here because this seems to be related to other ZA issues I have experienced recently. FYI I've seen a "Database is locked" message recently when trying to view the ZA dashboard.

From what I can tell ZA seems to be running.

Should I remove ZA and re-test?

Code Select Expand

***GOT REQUEST TO CHECK FOR UPDATES***
Currently running OPNsense 23.7 at Tue Aug  8 08:28:28 MST 2023
Fetching changelog information, please wait... done
Updating OPNsense repository catalogue...
Fetching meta.conf: . done
Fetching packagesite.pkg: .......... done
pkg: sqlite error while executing CREATE TABLE packages (id INTEGER PRIMARY KEY,origin TEXT,name TEXT NOT NULL,version TEXT NOT NULL,comment TEXT NOT NULL,desc TEXT NOT NULL,osversion TEXT,arch TEXT NOT NULL,maintainer TEXT NOT NULL,www TEXT,prefix TEXT NOT NULL,pkgsize INTEGER NOT NULL,flatsize INTEGER NOT NULL,licenselogic INTEGER NOT NULL,cksum TEXT NOT NULL,path TEXT NOT NULL,pkg_format_version INTEGER,manifestdigest TEXT NULL,olddigest TEXT NULL,dep_formula TEXT NULL,vital INTEGER NOT NULL DEFAULT 0);CREATE TABLE deps (origin TEXT,name TEXT,version TEXT,package_id INTEGER REFERENCES packages(id)  ON DELETE CASCADE ON UPDATE CASCADE,UNIQUE(package_id, name));CREATE TABLE categories (id INTEGER PRIMARY KEY, name TEXT NOT NULL UNIQUE );CREATE TABLE pkg_categories (package_id INTEGER REFERENCES packages(id)  ON DELETE CASCADE ON UPDATE CASCADE,category_id INTEGER REFERENCES categories(id)  ON DELETE RESTRICT ON UPDATE RESTRICT,UNIQUE(package_id, category_id));CREATE TABLE licenses (id INTEGER PRIMARY KEY,name TEXT NOT NULL UNIQUE);CREATE TABLE pkg_licenses (package_id INTEGER REFERENCES packages(id)  ON DELETE CASCADE ON UPDATE CASCADE,license_id INTEGER REFERENCES licenses(id)  ON DELETE RESTRICT ON UPDATE RESTRICT,UNIQUE(package_id, license_id));CREATE TABLE option (option_id INTEGER PRIMARY KEY,option TEXT NOT NULL UNIQUE);CREATE TABLE option_desc (option_desc_id INTEGER PRIMARY KEY,option_desc TEXT NOT NULL UNIQUE);CREATE TABLE pkg_option (package_id INTEGER NOT NULL REFERENCES packages(id) ON DELETE CASCADE ON UPDATE CASCADE,option_id INTEGER NOT NULL REFERENCES option(option_id) ON DELETE RESTRICT ON UPDATE CASCADE,value TEXT NOT NULL,PRIMARY KEY(package_id, option_id));CREATE TABLE pkg_option_desc (package_id INTEGER NOT NULL REFERENCES packages(id) ON DELETE CASCADE ON UPDATE CASCADE,option_id INTEGER NOT NULL REFERENCES option(option_id) ON DELETE RESTRICT ON UPDATE CASCADE,option_desc_id INTEGER NOT NULL REFERENCES option_desc(option_desc_id) ON DELETE RESTRICT ON UPDATE CASCADE,PRIMARY KEY(package_id, option_id));CREATE TABLE pkg_option_default (package_id INTEGER NOT NULL REFERENCES packages(id) ON DELETE CASCADE ON UPDATE CASCADE,option_id INTEGER NOT NULL REFERENCES option(option_id) ON DELETE RESTRICT ON UPDATE CASCADE,default_value TEXT NOT NULL,PRIMARY KEY(package_id, option_id));CREATE TABLE shlibs (id INTEGER PRIMARY KEY,name TEXT NOT NULL UNIQUE );CREATE TABLE pkg_shlibs_required (package_id INTEGER NOT NULL REFERENCES packages(id)  ON DELETE CASCADE ON UPDATE CASCADE,shlib_id INTEGER NOT NULL REFERENCES shlibs(id)  ON DELETE RESTRICT ON UPDATE RESTRICT,UNIQUE(package_id, shlib_id));CREATE TABLE pkg_shlibs_provided (package_id INTEGER NOT NULL REFERENCES packages(id)  ON DELETE CASCADE ON UPDATE CASCADE,shlib_id INTEGER NOT NULL REFERENCES shlibs(id)  ON DELETE RESTRICT ON UPDATE RESTRICT,UNIQUE(package_id, shlib_id));CREATE TABLE annotation (annotation_id INTEGER PRIMARY KEY,annotation TEXT NOT NULL UNIQUE);CREATE TABLE pkg_annotation (package_id INTEGER REFERENCES packages(id) ON DELETE CASCADE ON UPDATE RESTRICT,tag_id INTEGER NOT NULL REFERENCES annotation(annotation_id) ON DELETE CASCADE ON UPDATE RESTRICT,value_id INTEGER NOT NULL REFERENCES annotation(annotation_id) ON DELETE CASCADE ON UPDATE RESTRICT,UNIQUE (package_id, tag_id));CREATE TABLE pkg_conflicts (package_id INTEGER NOT NULL REFERENCES packages(id)  ON DELETE CASCADE ON UPDATE CASCADE,conflict_id INTEGER NOT NULL,UNIQUE(package_id, conflict_id));CREATE TABLE provides(    id INTEGER PRIMARY KEY,    provide TEXT NOT NULL);CREATE TABLE pkg_provides (package_id INTEGER NOT NULL REFERENCES packages(id)  ON DELETE CASCADE ON UPDATE CASCADE,provide_id INTEGER NOT NULL REFERENCES provides(id)  ON DELETE RESTRICT ON UPDATE RESTRICT,UNIQUE(package_id, provide_id));CREATE TABLE requires(    id INTEGER PRIMARY KEY,    require TEXT NOT NULL);CREATE TABLE pkg_requires (package_id INTEGER NOT NULL REFERENCES packages(id)  ON DELETE CASCADE ON UPDATE CASCADE,require_id INTEGER NOT NULL REFERENCES requires(id)  ON DELETE RESTRICT ON UPDATE RESTRICT,UNIQUE(package_id, require_id));PRAGMA user_version=2014; in file pkgdb.c:2333: attempt to write a readonly database
Unable to create repository OPNsense
Unable to update repository OPNsense
Updating SunnyValley repository catalogue...
Fetching meta.conf: . done
Fetching packagesite.pkg: .. done
pkg: sqlite error while executing CREATE TABLE packages (id INTEGER PRIMARY KEY,origin TEXT,name TEXT NOT NULL,version TEXT NOT NULL,comment TEXT NOT NULL,desc TEXT NOT NULL,osversion TEXT,arch TEXT NOT NULL,maintainer TEXT NOT NULL,www TEXT,prefix TEXT NOT NULL,pkgsize INTEGER NOT NULL,flatsize INTEGER NOT NULL,licenselogic INTEGER NOT NULL,cksum TEXT NOT NULL,path TEXT NOT NULL,pkg_format_version INTEGER,manifestdigest TEXT NULL,olddigest TEXT NULL,dep_formula TEXT NULL,vital INTEGER NOT NULL DEFAULT 0);CREATE TABLE deps (origin TEXT,name TEXT,version TEXT,package_id INTEGER REFERENCES packages(id)  ON DELETE CASCADE ON UPDATE CASCADE,UNIQUE(package_id, name));CREATE TABLE categories (id INTEGER PRIMARY KEY, name TEXT NOT NULL UNIQUE );CREATE TABLE pkg_categories (package_id INTEGER REFERENCES packages(id)  ON DELETE CASCADE ON UPDATE CASCADE,category_id INTEGER REFERENCES categories(id)  ON DELETE RESTRICT ON UPDATE RESTRICT,UNIQUE(package_id, category_id));CREATE TABLE licenses (id INTEGER PRIMARY KEY,name TEXT NOT NULL UNIQUE);CREATE TABLE pkg_licenses (package_id INTEGER REFERENCES packages(id)  ON DELETE CASCADE ON UPDATE CASCADE,license_id INTEGER REFERENCES licenses(id)  ON DELETE RESTRICT ON UPDATE RESTRICT,UNIQUE(package_id, license_id));CREATE TABLE option (option_id INTEGER PRIMARY KEY,option TEXT NOT NULL UNIQUE);CREATE TABLE option_desc (option_desc_id INTEGER PRIMARY KEY,option_desc TEXT NOT NULL UNIQUE);CREATE TABLE pkg_option (package_id INTEGER NOT NULL REFERENCES packages(id) ON DELETE CASCADE ON UPDATE CASCADE,option_id INTEGER NOT NULL REFERENCES option(option_id) ON DELETE RESTRICT ON UPDATE CASCADE,value TEXT NOT NULL,PRIMARY KEY(package_id, option_id));CREATE TABLE pkg_option_desc (package_id INTEGER NOT NULL REFERENCES packages(id) ON DELETE CASCADE ON UPDATE CASCADE,option_id INTEGER NOT NULL REFERENCES option(option_id) ON DELETE RESTRICT ON UPDATE CASCADE,option_desc_id INTEGER NOT NULL REFERENCES option_desc(option_desc_id) ON DELETE RESTRICT ON UPDATE CASCADE,PRIMARY KEY(package_id, option_id));CREATE TABLE pkg_option_default (package_id INTEGER NOT NULL REFERENCES packages(id) ON DELETE CASCADE ON UPDATE CASCADE,option_id INTEGER NOT NULL REFERENCES option(option_id) ON DELETE RESTRICT ON UPDATE CASCADE,default_value TEXT NOT NULL,PRIMARY KEY(package_id, option_id));CREATE TABLE shlibs (id INTEGER PRIMARY KEY,name TEXT NOT NULL UNIQUE );CREATE TABLE pkg_shlibs_required (package_id INTEGER NOT NULL REFERENCES packages(id)  ON DELETE CASCADE ON UPDATE CASCADE,shlib_id INTEGER NOT NULL REFERENCES shlibs(id)  ON DELETE RESTRICT ON UPDATE RESTRICT,UNIQUE(package_id, shlib_id));CREATE TABLE pkg_shlibs_provided (package_id INTEGER NOT NULL REFERENCES packages(id)  ON DELETE CASCADE ON UPDATE CASCADE,shlib_id INTEGER NOT NULL REFERENCES shlibs(id)  ON DELETE RESTRICT ON UPDATE RESTRICT,UNIQUE(package_id, shlib_id));CREATE TABLE annotation (annotation_id INTEGER PRIMARY KEY,annotation TEXT NOT NULL UNIQUE);CREATE TABLE pkg_annotation (package_id INTEGER REFERENCES packages(id) ON DELETE CASCADE ON UPDATE RESTRICT,tag_id INTEGER NOT NULL REFERENCES annotation(annotation_id) ON DELETE CASCADE ON UPDATE RESTRICT,value_id INTEGER NOT NULL REFERENCES annotation(annotation_id) ON DELETE CASCADE ON UPDATE RESTRICT,UNIQUE (package_id, tag_id));CREATE TABLE pkg_conflicts (package_id INTEGER NOT NULL REFERENCES packages(id)  ON DELETE CASCADE ON UPDATE CASCADE,conflict_id INTEGER NOT NULL,UNIQUE(package_id, conflict_id));CREATE TABLE provides(    id INTEGER PRIMARY KEY,    provide TEXT NOT NULL);CREATE TABLE pkg_provides (package_id INTEGER NOT NULL REFERENCES packages(id)  ON DELETE CASCADE ON UPDATE CASCADE,provide_id INTEGER NOT NULL REFERENCES provides(id)  ON DELETE RESTRICT ON UPDATE RESTRICT,UNIQUE(package_id, provide_id));CREATE TABLE requires(    id INTEGER PRIMARY KEY,    require TEXT NOT NULL);CREATE TABLE pkg_requires (package_id INTEGER NOT NULL REFERENCES packages(id)  ON DELETE CASCADE ON UPDATE CASCADE,require_id INTEGER NOT NULL REFERENCES requires(id)  ON DELETE RESTRICT ON UPDATE RESTRICT,UNIQUE(package_id, require_id));PRAGMA user_version=2014; in file pkgdb.c:2333: attempt to write a readonly database
Unable to create repository SunnyValley
Unable to update repository SunnyValley
Error updating repositories!
pkg: Repository OPNsense cannot be opened. 'pkg update' required
pkg: Cannot get an advisory lock on a database, it is locked by another process
***DONE***

[sy]

Hi,

It seems that pkg can not get OPNsense repo. What if you run pkg update -f command on the console?

[doug_phoenix]

Hi @sy,

Thank you. I was able to complete the package update from the console.

Unfortunately, now my network is down. I can connect to the OPNsense UI, but not the ZA UI. Internet/WAN and other LAN devices are down.

I've rebooted twice (from the console - reboot did not work from the UI).

I'm going to reinstall OPNsense again. I think it will work without ZA.

I do appreciate your kind support!

[sy]

Hi,

Is there any news on the issue?

[doug_phoenix]

Hi @sy,

Sorry, I missed your message.

I experienced another database issue that pointed to a likely SSD problem.
https://forum.opnsense.org/index.php?topic=35360.0

So I replaced the SSD with one provided by the OEM. Currently running 23.7.1_3 along with AdGuard Home and Crowdsec. So far the system looks stable and smartctl reports a healthy SSD.

I'd like to try ZA again, but with all of my issues I would like to confirm that my hardware is capable and that the new release is stable. How can I test my system? Should I try the previous version of ZA or has this one stabilized?

Protectli VP2410
Intel Celeron J4125
16 GB DRAM
480 GB M.2 SSD
coreboot BIOS

It's the same config as reviewed here: https://homenetworkguy.com/review/protectli-vp2410/
FYI the review shows good throughput using ZenArmor (older version, clearly).

Thanks again for your great support.

Edit: I've learned that there is a conflict between ZA and "Elasticsearch from the mimugmail repository" (which I'm using for AdGuard Home). ZA suggests using the repository database as a remote Elesticsearch database, but it appears there are issues. See:
https://github.com/mimugmail/opn-repo/issues/116

So, if I were to try ZA again, it seems the easy solution would be to remove mimugmail and AdGuard Home. :(

[doug_phoenix]

I went forward with removal of AdGuard and Mimugmail repository. Installed ZA and all seems well.

Even updated packet engine a couple of moments ago. Will hold off on OPNSense 23.7.2 for a little while.

So, it seems that updating OPNSense and ZA exposed an SSD hardware issue.  FYI I was running a WD Blue 500 SATA III M.2 SSD. Less than 6 months old!