High CPU usage when downloading

[fox983]

Hi all,
when downloading big files from internet the firewall's CPU goes to 90 - 100% on APU2C4 (with Intel network cards).

SYSTEM: DIAGNOSTICS: ACTIVITY
root   93   0   43M   28M   CPU2   2   795:57   76.86%   /usr/local/bin/python3 /usr/local/opnsense/scripts/netflow/flowd_aggregate.py (python3.9)

I've read many topic (eg. https://forum.opnsense.org/index.php?topic=31999.0), tried to clear logs and disable Netflow but no luck.
Any ideas?

[dMopp]

Hardware offload active ?

[axsdenied]

If you're not experience any performance issues, I'd just leave it alone.  If this isn't the norm for you and something has changed, we'll need more data to help diagnose.

[fox983]

Hardware offload active ?

No

[fox983]

If you're not experience any performance issues, I'd just leave it alone.  If this isn't the norm for you and something has changed, we'll need more data to help diagnose.

It happens mostly with torrents, I'm trying to download 4 Linux Mint and the CPU usage is at 100%

[doktornotor]

Get some better HW, or move on.

[fox983]

 :)

[Seimus]

Hi all,
when downloading big files from internet the firewall's CPU goes to 90 - 100% on APU2C4 (with Intel network cards).

SYSTEM: DIAGNOSTICS: ACTIVITY
root   93   0   43M   28M   CPU2   2   795:57   76.86%   /usr/local/bin/python3 /usr/local/opnsense/scripts/netflow/flowd_aggregate.py (python3.9)

I've read many topic (eg. https://forum.opnsense.org/index.php?topic=31999.0), tried to clear logs and disable Netflow but no luck.
Any ideas?

"when downloading big files from internet"

Whats your throughput?
APU uses really very very very old SOC (I have one APU at home) and I can tell you such behaviors are kinda now normal on that SOC and more prominent the more throughput + features configured you have.

APU performs OK with OPNsense at 200Mbit/s with around 6VLANs + Shaper + basic rules. Anything above that and you will see performance degradation.

Regards,
S.

[fox983]

Hi all,
when downloading big files from internet the firewall's CPU goes to 90 - 100% on APU2C4 (with Intel network cards).

SYSTEM: DIAGNOSTICS: ACTIVITY
root   93   0   43M   28M   CPU2   2   795:57   76.86%   /usr/local/bin/python3 /usr/local/opnsense/scripts/netflow/flowd_aggregate.py (python3.9)

I've read many topic (eg. https://forum.opnsense.org/index.php?topic=31999.0), tried to clear logs and disable Netflow but no luck.
Any ideas?

"when downloading big files from internet"

Whats your throughput?
APU uses really very very very old SOC (I have one APU at home) and I can tell you such behaviors are kinda now normal on that SOC and more prominent the more throughput + features configured you have.

APU performs OK with OPNsense at 200Mbit/s with around 6VLANs + Shaper + basic rules. Anything above that and you will see performance degradation.

Regards,
S.

My throughput is about 170Mbit/s, 2 VLANs, Shaper, Monit, Wireguard and 15/20 rules. In your signature you have APU2D2, mine is APU2C4. So do you think this is normal behavior?
Thank you

[Seimus]

APU2D2 and C4 are having the exact same SOC, differences are mostly in NIC and onboard features. I would say this is normal. Shaper is CPU hungry. If you need to Shape the more BW you have the more the Shaper will ask to eat for example. Monit as an example is another resource drainer similiar to shaper the more you push the more you need to give him.

Try to disable Shaper and Monit and you should see APu is performing bit better.

When I was doing testing on APU with iperf interVLAN I has able to get on 50 parallel streams around 500-650Mbit (700-800 if I did some tuning) at MAX without shaping, but at the cost of CPU at 100% and 15.5 load cycles per 5s interval. This was done using 2 VLANs and 2 simple rules each per interface.

In year 2023 if we are using APUs, we need to be more modest on which feature to use.

Another points is APUs will be discontinued as announced by the manufacturer.

APUs are great devices, but they long outlived they usefulness for modern Network devices the more BW you have the more performance issues are prominent. That was as well the reason I switched to a newer SOC. I need to be able reach the BW provided by ma ISP yet I need to have all my features implemented working correctly.

Regards,
S.

[fox983]

Thank you for sharing this!
Maybe next year I will buy new hardware.
Regards