PC Engines APU2 1Gbit traffic not achievable

[thowe]

Just to be sure:
- You test with iperf THROUGH the firewall. I.e. iperf is not running on the firewall but on separate hosts "on both sides" of the firewall?
- You have only set pf rules but no other services like IDS, IDP, Sensei etc.

[thowe]

Something you can check: configure powerd to use "Maximum" instead of "Hiadaptive".

As discussed here: https://forum.opnsense.org/index.php?topic=21194.msg99228#msg99228

[Patrick M. Hausen]

If you use "Maximum", anyway, just disable powerd.

[Ricardo]

Are there any penalties from not using powerd (otherwise said to not enable the powerd service)? I assume some hidden bugs may surface, if some system strongly assumes powerd is an always present component. Or at least I would be more careful to say too quickly that powerd is not necessary at all, and doesnt cause any issues.

[Patrick M. Hausen]

Code Select Expand

NAME
     powerd – system power control utility

SYNOPSIS
     powerd [-a mode] [-b mode] [-i percent] [-m freq] [-M freq] [-N]
            [-n mode] [-p ival] [-P pidfile] [-r percent] [-s source] [-v]

DESCRIPTION
     The powerd utility monitors the system state and sets various power
     control options accordingly.  It offers power-saving modes that can be
     individually selected for operation on AC power or batteries.
[...]


Powerd's only function in FreeBSD is to set the CPU to power saving modes when idle. There is nothing else that depends on powerd. You do not need to run it.

[Ricardo]

Somebody complained that their APU2 CPU got stuck at 600Mhz, and had to actually enable the powerd to force CPU clockspeed go over that 600Mhz.
As I said, the behavior of a system may not be easy to predict, even if the theory says something, reality may be something different.

[dave]

Are there any penalties from not using powerd (otherwise said to not enable the powerd service)? I assume some hidden bugs may surface, if some system strongly assumes powerd is an always present component. Or at least I would be more careful to say too quickly that powerd is not necessary at all, and doesnt cause any issues.

Disabling PowerD and enable the core performance boost via the bios will lock the cores at 1.4Ghz.
The APU's are only ~10w devices, so you don't need to worry about power savings \ heat.

[rmayr]

Update: I found the culprit for the drop of more than 1/3 in throughput: just enabling IPSec (with configured tunnels up and running) drops locally routed performance from 750-800Mbps to 500Mbps for traffic that doesn't go through the tunnel. This is using IPSec policies and not with a virtual network device.

[rmayr]

And just to confirm: yes, there are two hosts on different sides of the box, one iperf3 server, one client.

coreboot has been updated to the latest available version. PowerD is running and normally set to Hiadaptive as I actually want to save some power for most of the time when there is little traffic. A quick comparison doesn't seem to show a measurable difference between Hiadaptive and Maximum, though performance drops when I disable PowerD altogether (probably confirming the suspicion that the CPU is stuck at 600MHz without it running).

[rmayr]

Further datapoints: Having flowd_aggregate running (with all local VLAN interfaces monitored) drops around 50Mbps throughput when samplicate is stopped and about 250Mbps when both are running. But this part is - if not good - than at least explainable, as it certainly adds CPU load. The IPSec related throughput drop for streams not hitting IPSec tunnels (which stacks with the netflow drop, i.e. when both are enabled, I only average around 250Mbps total throughput) is what puzzles me.

[Ricardo]

check this forum for the "APU2 stuck at 600Mhz" issue:

https://github.com/pcengines/coreboot/issues/457

Regarding the policy based ipsec enablement immediately halves the throughput even if the traffic is bypassing the vpn tunnel, is very concerning. I also have some policy based vpn tunnels, so it may further limit my WAN speed, even if that traffic is not getting routed into the vpn tunnel. Big mess, I have to say, and years can pass by without resolution :(

[Patrick M. Hausen]

Somebody complained that their APU2 CPU got stuck at 600Mhz, and had to actually enable the powerd to force CPU clockspeed go over that 600Mhz.
As I said, the behavior of a system may not be easy to predict, even if the theory says something, reality may be something different.

I was involved in that discussion and IMHO we came to the conclusion that you needed to disable powerd to get beyond 600 MHz. That's precisely why I recommend that.

[thowe]

I have opened the ticket with the maintainers of Coreboot for the PC Engines boards. So far, however, there has been nothing further.

I was able to solve the problem by keeping PowerD enabled but setting the mode to maximum. Since then I didn't have the problem anymore.

And if the CPU was limited to 600MHz it cost a lot of performance. In my setup I just got away with it - but had zero headroom. When the CPU is running normally the utilization is rarely more than 50%.

[rmayr]

Regarding the policy based ipsec enablement immediately halves the throughput even if the traffic is bypassing the vpn tunnel, is very concerning. I also have some policy based vpn tunnels, so it may further limit my WAN speed, even if that traffic is not getting routed into the vpn tunnel. Big mess, I have to say, and years can pass by without resolution :(

Indeed. This happens not only for LAN->WAN traffic, but also for traffic between two different internal (e.g. LAN and DMZ) segments with no NAT involved and only directly connected routes in use. I have not yet tried with VTI instead of policy based IPsec, but this issue may make OpnSense a non-starter for the intended production use at our university institute (that is the reason why I am now spending far too much time putting OpnSense through such tests).

[mimugmail]

Regarding the policy based ipsec enablement immediately halves the throughput even if the traffic is bypassing the vpn tunnel, is very concerning. I also have some policy based vpn tunnels, so it may further limit my WAN speed, even if that traffic is not getting routed into the vpn tunnel. Big mess, I have to say, and years can pass by without resolution :(

Indeed. This happens not only for LAN->WAN traffic, but also for traffic between two different internal (e.g. LAN and DMZ) segments with no NAT involved and only directly connected routes in use. I have not yet tried with VTI instead of policy based IPsec, but this issue may make OpnSense a non-starter for the intended production use at our university institute (that is the reason why I am now spending far too much time putting OpnSense through such tests).

You really want to run a university institute in production with a APU device??  :o