Netflow + insight local cache and CPU utilization

[Ricardo]

Hello all,

I am trying to find any documentation about the storage requirement for Netflow + the Insight collector, and the CPU load it generates to deal with this type of workload.

If I recall correctly, I read somewhere (unfortunately I am unable to find it anymore, where exactly) that the max storage that the /var/netflow/ files occupy should be max. around 100 MB. In contrast, for the moment it is about total of 700 MB after a week of run. There is 1 file (I cant recall its name) which is 500+ MB, the rest are much smaller.
I use tmpfs on /var, and also tried to disable the backup of netflow data during reboot, as it takes ages to backup this 700+ MB before shutdown and restore again when it boots, plus it wears my tiny 16 GB SSD.

The second part of my question: there is a periodic load spike, when a python process eats 1 of the 4 cores for 5-6 seconds, then goes down to 0. After 10-12 seconds, it is repeated. Happens during low traffic periods as well. I assume that python process has to do something with this Netflow/Insight workload. Is it taking such huge CPU load due to the large 700 MB+ database, and should decrease if the database is much smaller (e.g. after a fresh reboot with 0 database content)?