Messages

If you use Netflow reporting, you will have /var/log/flowd*.log which are rotated and thus they do not take up much space, but get written A LOT.

There is a setting to have /var/log in the RAM disk to avoid this - you need to reboot to activate it.
With that setting, all logs will get lost on every reboot.

Also, the free space not diminishing is a tell-tale sign of a database that is being written to. In my case, it was Zenarmor, which uses an Sqlite database that was constantly being written to.

Having logs written to a RAM disk that will be flushed in the event of a power outage is obviously not ideal.

At the moment we are at 5.64 TB written to the drive and I am wondering if the scale of writes can be concidered to be in the range of normal? At this rate the reliability of the SSD drive is in question since it is only rated for 160TB of writes within the warranty and possibly more before wearing out completely.

If there are features like this that will wear out the storage medium at this pace, I would like to have only these logs that are perhaps not very important to be held on a ram disk.

I have now disabled the collection of flowd logs and will see if this has an effect on the issue.

It has been 11 days and the writes are at 4.37TB. That makes it 79GB/day of writes that do not seem to take up any disk space.

https://imgur.com/a/ZLZvAet

Still no explanation as to what is causing this.

https://imgur.com/a/Rk1d1FY

We have been running an installation of OPNsense with the latest version for a few months and have noticed alarming levels of SSD writes that we cannot explain.

There is very little logging happening. The file system has very little storage used and we have zenarmor and netflow enabled but not much more. Most firewall rules are set to not log traffic.

Here are two screenshots of SSD attributes about 30 days apart. The writes have increased by 2,11TB in that time.
https://imgur.com/a/SUzqITC

Ramdisks are not in use and only 13GB of storage is used currently

https://imgur.com/a/bnnq66A

When running the iostat command in shell, it does not show high writes:

Code Select Expand

iostat -x 1 10
                        extended device statistics
device       r/s     w/s     kr/s     kw/s  ms/r  ms/w  ms/o  ms/t qlen  %b
nvd0           0      46      1.7    852.3     0     0     1     0    0   0
md43           0       0      0.0      1.8     0     0     0     0    0   0
                        extended device statistics
device       r/s     w/s     kr/s     kw/s  ms/r  ms/w  ms/o  ms/t qlen  %b
nvd0           0     424      0.0   9191.4     0     0     1     0    0   1
md43           0       2      0.0      8.0     0     0     0     0    0   0
                        extended device statistics
device       r/s     w/s     kr/s     kw/s  ms/r  ms/w  ms/o  ms/t qlen  %b
nvd0           0       6      0.0    103.9     0     0     1     0    0   0
md43           0       0      0.0      0.0     0     0     0     0    0   0
                        extended device statistics
device       r/s     w/s     kr/s     kw/s  ms/r  ms/w  ms/o  ms/t qlen  %b
nvd0           0       4      0.0     24.0     0     0     1     1    0   0
md43           0       0      0.0      0.0     0     0     0     0    0   0
                        extended device statistics
device       r/s     w/s     kr/s     kw/s  ms/r  ms/w  ms/o  ms/t qlen  %b
nvd0           0       7      0.0    100.0     0     0     1     0    0   1
md43           0       0      0.0      0.0     0     0     0     0    0   0
                        extended device statistics
device       r/s     w/s     kr/s     kw/s  ms/r  ms/w  ms/o  ms/t qlen  %b
nvd0           0      45      0.0   2585.2     0     0     1     0    0   1
md43           0       0      0.0      0.0     0     0     0     0    0   0
                        extended device statistics
device       r/s     w/s     kr/s     kw/s  ms/r  ms/w  ms/o  ms/t qlen  %b
nvd0           0     251      0.0   8193.3     0     0     4     0    0   6
md43           0       1      0.0      4.0     0     0     0     0    0   0
                        extended device statistics
device       r/s     w/s     kr/s     kw/s  ms/r  ms/w  ms/o  ms/t qlen  %b
nvd0           0       4      0.0     32.0     0     9    14    12    0   6
md43           0       0      0.0      0.0     0     0     0     0    0   0
                        extended device statistics
device       r/s     w/s     kr/s     kw/s  ms/r  ms/w  ms/o  ms/t qlen  %b
nvd0           0       4      0.0     56.0     0     0    12     6    0   5
md43           0       0      0.0      0.0     0     0     0     0    0   0
                        extended device statistics
device       r/s     w/s     kr/s     kw/s  ms/r  ms/w  ms/o  ms/t qlen  %b
nvd0           0       0      0.0      0.0     0     0     0     0    0   0
md43           0       0      0.0      0.0     0     0     0     0    0   0
What could be causing this?

We have the exact same issue running 23.1 with an Watchguard M370 appliance.

Lan port appears up but the connectivity is lost and it is not visible from the lat network even with arp -a.

The problem happens infrequently every 7-14 days and is very difficult to track down. VPN and WAN interface work and the firewall management is acccessible when this happens (Through VPN). Zenarmor is activated, but it is not really doing much besides reporting: Routed Mode (L3 Mode, Reporting + Blocking) with native netmap driver.

Will try with the emulated driver if that will fix the issue. The logs have nothing noteworthy from the time of the issue happening.

Just installed the latest 23.1.6 patches but not feeling optimistic since this has happened multiple times already.

Any ideas on tracking down the issue?

[Here is the procedure to get the utility compiled on FreeBSD]

There is a fan control utility for Watchguard firewalls called  WGXepc

There is a writeup on it here: https://www.hexhound.com/quiet-the-fan-on-your-pfsense-watchguard-firewall/ and it seems to be directly compatible with pfsense.

Tried to get it running on OPNsense and the precompiled binary does not run on openbsd.

Here is the procedure to get the utility compiled on FreeBSD

This has been tested with OPNsense 22.1.8_1-amd64 running FreeBSD 13.0-STABLE
with Watchguard Firebox M370 hardware


SSH into your OPNsense box with your favorite client

Download the source  https://raw.githubusercontent.com/stephenw10/WGXepc/mastcode for the utility from github: https://github.com/stephenw10/WGXepc

Code Select Expand

fetch  https://raw.githubusercontent.com/stephenw10/WGXepc/master/WGXepc.c

Compile the code

Code Select Expand

cc WGXepc.c -o WGXepc

Change permissions to be able to run the binary

Code Select Expand

chmod 700 WGXepc

You can now run the binary and control fans etc.

Code Select Expand

./WGXepc
Found Firebox M370/470/570/670.
WGXepc Version 1.6_1 22/11/2020 stephenw10
WGXepc can accept two arguments:
-f (CPU fan) will return the current and minimum fan speed or if followed
    by a number in hex, 00-FF, will set it.
-f2 (System fan) will return the current and minimum fan speed or if followed
    by a number in hex, 00-FF, will set it.
-l (led) will set the arm/disarm led state to the second argument:
    red, green, red_flash, green_flash, red_flash_fast, green_flash_fast, off
-b (backlight) will set the lcd backlight to the second argument:
    on or off. Do not use with LCD driver.
-t (temperature) shows the current CPU temperature reported by the
    SuperIO chip. X-e box only.
Not all functions are supported by all models


You can control fan speed from 0 to 99 by running

Code Select Expand

./WGXepc -f 10

Could some one help to get the control scripts and boot time running working, because I am not yet that familiar with the inner workings of OPNsense.