Wireguard speed on OPNsense and PFsense

[mgiammarco]

Hello,
I have made two identical hetzner VMs, one with OPNsense 21.1.3 and one with PFsense 2.5
I have tried wireguard performance:
- PFsense wireguard saturates my client with 600mbit/s
- OPNsense wireguard reaches only 40mbits with 100% cpu on OPNsense.

I ask:
- is it due because OPNsense version is not in kernel?
- is it due because I have not correctly enabled aes-ni?
- what can I do?

Thanks,
Mario

[chemlud]

First question you should answer to yourself: Do you want to run pfSense 2.5 with WireGuard at all, considering the quality of the kernel code pushed forward by Netgate?

https://arstechnica.com/gadgets/2021/03/buffer-overruns-license-violations-and-bad-code-freebsd-13s-close-call/

My answer is: Definitely NO...

[mgiammarco]

I perfectly know what happened in PFsense but I have asked another thing...

[franco]

Context switches are the issue. Depending on the CPU throughput you get very bad results. If you can get a faster CPU you should, because this one seems rather slow.


Cheers,
Franco

[MartB]

Yeah 100% hardware on your end, im running 400+ mbit/s on a Intel(R) Celeron(R) J4115 CPU @ 1.80GHz (4 cores)
) with wireguard-go.
Just wait until the proper implementation is merged to opnsense, the wireguard authors are working on fixing the mess that netgate financed.

[mgiammarco]

In my virtual machine I have two cores of Intel Xeon Skylake IBRS, it does not seem to me a cpu so slow compared to a J4115.
@franco if @MartB is able to reach 400mbits it seems that context switches is not the only problem

[franco]

I'm not sure what you are implying. This is simple. Just be sure the host CPU is only serving your guest for reliable measurements.

At my last job a customer complained about sluggish appliance performance. It was a VM on a host that had just over 100 VMs running at the same time. I won't forget scrolling through the actual VM list in the teams session in disbelief.


Cheers,
Franco

[mgiammarco]

Apart the fact on same vm I got 600mbit with PFsense now I have rebuilt the VM with fully dedicated 8 core high performance xeon. The VM now costs 10 times more.
Guess what? Peak transfer now 54, not 40....
But it is common to bsd forums... laugh about people hardware to avoid give real replies.

[Maurice]

Just to add another data point:

OPNsense, Hyper-V VM, 2 virtual cores, Intel Core i5-2520M (10yo mobile CPU!), three other VMs on the same CPU. WireGuard throughput 150+ Mbps. And the CPU might not even be the limiting factor, not sure. WAN is only 200 Mbps.

Cheers

Maurice

[franco]

But it is common to bsd forums... laugh about people hardware to avoid give real replies.

Please stop with the self-pity. We're not laughing. You provide irreproducible data points and doing so surely found a problem with your specific testbed but nobody will be able to help you if you keep waving the WireGuard kernel module flag as a measurement point to raise concerns for improvement.

WHAT should be improved? The go performance? That IS what the kernel module is supposed to do.

If you see below-average results in your go testing YOU need to find out WHY in YOUR setup first for US to be able to HELP.


Cheers,
Franco

[MartB]

You might give the new if_wg.ko on 21.1.4 a try
https://forum.opnsense.org/index.php?topic=20978.msg106200#msg106200

Please report back with some speedtest results and system resource usage numbers.