Python 3.7 using max memory and filling disk space

[opnfwb]

I noticed this morning that Python 3.7 seems to be maxing out my memory and is filling up the disk space on my SSD. I have not made any config changes to the firewall for the last week, the last config change that I made a week ago was to change the firewall optimization from "standard" to "aggressive". The firewall has been very stable for weeks without rebooting until today. It is currently passing traffic for now.

Is there a reason why Python 3.7 would fill the disk like this? This seems like a major problem in how it is functioning.

Firewall specs are as follows:
Intel J3455 (bare metal install)
16GB RAM
120GB SSD
Dual Broadcom NICs
OPNsense 19.7.7-amd64
FreeBSD 11.2-RELEASE-p16-HBSD
OpenSSL 1.0.2t 10 Sep 2019

Below are the screenshots showing the problem.

[opnfwb]

Replying 2nd time to add more attachments due to size limits.

[opnfwb]

Looks like Python is completely filling the disk. I tried restarting all of the services shown on the dashboard homepage, this did not have any impact on the disk fill. I'm shutting down the firewall now, these are the last screen caps I grabbed before shutdown.

[AdSchellevis]

update tables is used for firewall aliases, could be caused by an alias which is  fetching a very large (?) list. I would inspect aliases and their settings first (check if sources are valid, the log might show more details as well).

[opnfwb]

Thank you for the response. This is a fairly basic install and I don't use aliases. Here's a screenshot of my firewall aliases table.

The only packages I have installed are wake-on-lan and smart tools. I'm not using Suricata or Sensei on this install.

[AdSchellevis]

ok, that's odd, does this file exist? /usr/local/etc/filter_tables.conf (if so, what does it look like)

[opnfwb]

Here's the output of my filter_tables.conf file.

[AdSchellevis]

ok, empty is good, although the script should be finished quite rapidly in that case.

What happens is you execute

Code: [Select]

/usr/local/opnsense/scripts/filter/update_tables.py manually from a console?

Just to be sure, this is a standard installation, didn't try to add any packages using other sources then ours and didn't use pip from a console, right?

[opnfwb]

When I manually run that command I receive back "status" : "ok"

This is a stock install, I think I started with an early version of 19.7 and have upgraded it as updates were made available through the OPNsense repos. I have not made any customizations to packages and have only installed two packages available on the mirrors, WOL and Smart.

One thing I also wanted to add, the "python3.7.core" file that was filling the disk was not removed on a reboot. I shutdown the firewall gracefully and powered it back up. When it came up, the disk space was completely exhausted and I had to manually remove the "python3.7.core" file to get the space back.

[AdSchellevis]

Doesn't sound very logical, if there are no aliases, the script exits almost directly (without consuming a lot of memory). Can you perform a health check on the machine (System -> Firmware and then click the button on the top right)

[opnfwb]

I did a System/Firmware/Health audit and below are the results of this:

Code: [Select]

***GOT REQUEST TO AUDIT HEALTH***
>>> Check installed kernel version
Version 19.7.7 is correct.
>>> Check for missing or altered kernel files
No problems detected.
>>> Check installed base version
Version 19.7.7 is correct.
>>> Check for missing or altered base files
No problems detected.
>>> Check for and install missing package dependencies
Checking all packages: .......... done
>>> Check for missing or altered package files
Checking all packages: .......... done
***DONE***
At this point I'm stumped. Not sure what caused this? I've run memcheck on the firewall as well and have no errors, so I don't think its hardware related. It has been stable for months until the odd Python CPU usage and disk fill yesterday.

[AdSchellevis]

Haven't seen this before either. The specific script doesn't do a lot without alias entries in the xml file.