Home
Help
Search
Login
Register
OPNsense Forum
»
Archive
»
19.1 Legacy Series
»
flowd.log not rotated
« previous
next »
Print
Pages: [
1
]
Author
Topic: flowd.log not rotated (Read 4118 times)
Andreas_
Jr. Member
Posts: 63
Karma: 1
flowd.log not rotated
«
on:
May 31, 2019, 10:14:12 am »
I've been running netflow logging a while now flawlessly. Since about two weeks (I guess post-19.1.5), rotation of flowd.log stalls, resulting in a fillled disk and consequently stopping some services for out-of-diskspace reasons. This includes flowd and flowd_aggregate.py. The firewall is monitored, so I could check that flowd_aggregate is killed from out-of-disk, not before.
I upgraded to 19.1.8, and see some strange behaviour of flowd_aggregate:
Usually, flowd.log will reach between 11MB and 13MB before rotating; it takes about 5 minutes for this size. But I also can observe this:
-rw------- 1 root wheel 3.2M May 31 10:01 flowd.log
-rw------- 1 root wheel 12M May 31 10:00 flowd.log.000001
-rw------- 1 root wheel 13M May 31 09:56 flowd.log.000002
-rw------- 1 root wheel 56M May 31 09:51 flowd.log.000003
-rw------- 1 root wheel 13M May 31 09:29 flowd.log.000004
-rw------- 1 root wheel 23M May 31 09:24 flowd.log.000005
-rw------- 1 root wheel 12M May 31 09:15 flowd.log.000006
-rw------- 1 root wheel 13M May 31 09:11 flowd.log.000007
-rw------- 1 root wheel 11M May 31 09:06 flowd.log.000008
-rw------- 1 root wheel 12M May 31 09:01 flowd.log.000009
-rw------- 1 root wheel 12M May 31 08:56 flowd.log.000010
So obviously flowd_aggregate stalls sometimes for some minutes, and then continues to work. I can't see any anomalies in cpu load or usage during this period. Nothing in the system log, the last flowd related message is
May 31 06:47:26 fw05a flowd_aggregate.py: vacuum done
While this is not the total stop of flowd.log rotation I've been suffering from in the last weeks, it still seems suspicious to me. What's going wrong here?
Added: stalling is happening right now, and I can see the flowd_aggregate proces consuming a lot of CPU.
«
Last Edit: May 31, 2019, 10:17:09 am by Andreas_
»
Logged
Andreas_
Jr. Member
Posts: 63
Karma: 1
Re: flowd.log not rotated: aggregation too slow
«
Reply #1 on:
May 31, 2019, 01:43:13 pm »
I started with fresh logging, i.e. deleted /var/log/flowd* and /var/netflow/*, restarted flowd and running flowd_aggregate.py with some timing added in the console.
I can see that a typical aggregation run will take 70-80 seconds, then rotation is checked, then 15 seconds nap.
This means, that currently the flowd_aggregate process is at 85% of its capacity; there are certainly phases where firewall traffic is heavier than currently.
flowd.log is filled with a rate of about 2.5-5MB/min, /var/netflow/* has about 5MB. Doesn't seem to heavy to me.
Machine as a 8-core C3758CPU, Load avg about 1.3, with CPU at some % (flowd_aggregate will add 12%), SATA-DOM SSD.
Logged
Print
Pages: [
1
]
« previous
next »
OPNsense Forum
»
Archive
»
19.1 Legacy Series
»
flowd.log not rotated