Netflow - continous 50-60 Mb/s disk writes

Started by molnart, Today at 02:35:35 PM

Previous topic - Next topic
I have a long running issue with netflow - literally for years. netflow starts to hog up my disk and cpu usage with huge amount of writes. restarting opnsense takes 20-30 minutes because of backing up and restoring netflow data.
Usually its sqlite accessing some .sqlite.fix file.
Every time this happens, i need to reset netflow logs from the UI, manually stop the sqlite process and delete the contents of the netflow directory.

Is there a better way to handle this, while keeping long term netflow statistics (10+ years)? 

Do you use a ZFS install? Things like that usually happen when there are filesystem inconsistencies created during hard stops of the netflow process. That cannot happen with ZFS.
Intel N100, 4* I226-V, 2* 82559, 16 GByte, 500 GByte NVME, ZTE F6005

1100 down / 450 up, Bufferbloat A+

Send the netflow data to a different system.
Deciso DEC750
People who think they know everything are a great annoyance to those of us who do. (Isaac Asimov)