Upgrade to 20.7 , huge memory use (from 2Gb to 11Gb) Firewall log files Dead !

[cardonarico]

Hi there, I have been a happy user of OpnSense for the last 2 years.
All upgrades went always smooth with huge improvements every upgrade.
However from 20.1 to 20.7 things just started to go south.
Right now, a quiet and functional firewall ( Corei3 16Gb Ram, Intel NIC , SSD) has showing issues right after the upgrade.
All firewall log files  ( Live view, Overview, Plain View) are completely dead or extremely low response.
When it works it takes 5-10 minutes to show data what makes them useless.
Also the memory use skyrocket (from usually 2-3Gb to 10-12Gb)
Finally mailtrail and Suricata are behaving quite odd , all services are up, logs for both maitrail and suricata seems ok but, logs are low ( on Suricata case , none, no detection at all )

Also , the services from time to time goes down (Default gateway checker, NTP server , etc) what have to be restarted manually.
Is there anybody else experiencing such behaviour ? any suggested actions ? I am completely lost 

Thanks guys

[alexharvard]

Hi there,
i can confirm this issue from cardonarico with the firewall logs.
On my tests the logs are working correct right after reboot for about 20 to 60 seconds and stop then.
Tested on AMD GX-412TC SOC with 4Gb Memory and 42GB SSD.

[aimdev]

I had some issues on an upgrade, see

https://forum.opnsense.org/index.php?topic=18358.0

[cloudz]

I can +1 on the logs and Live view. They stopped updating on Aug 1. Right on the update moment.
No issues with memory or disk usage however. I'm not using suricata / ..

[mimugmail]

Hi there, I have been a happy user of OpnSense for the last 2 years.
All upgrades went always smooth with huge improvements every upgrade.
However from 20.1 to 20.7 things just started to go south.
Right now, a quiet and functional firewall ( Corei3 16Gb Ram, Intel NIC , SSD) has showing issues right after the upgrade.
All firewall log files  ( Live view, Overview, Plain View) are completely dead or extremely low response.
When it works it takes 5-10 minutes to show data what makes them useless.
Also the memory use skyrocket (from usually 2-3Gb to 10-12Gb)
Finally mailtrail and Suricata are behaving quite odd , all services are up, logs for both maitrail and suricata seems ok but, logs are low ( on Suricata case , none, no detection at all )

Also , the services from time to time goes down (Default gateway checker, NTP server , etc) what have to be restarted manually.
Is there anybody else experiencing such behaviour ? any suggested actions ? I am completely lost 

Thanks guys

Some steps to find the issue:
- put a monitor on the firewall and check logs while rebooting, any syslog crash?
- What about system.log? Is there anything to see?
- Can you login via SSH and do a "clog -f /var/log/filter.log"? What happens?
- It would be good to stop maltrail and Suricata and check if memory still goes up, if not, only start one of them to rule out which one eats memory

[heyudude]

I had the same issue: I restarted the syslogd service and all was fine!