Messages

I regret falling silent in the middle of this discussion (I had to move abroad), but I finally got around to tinkering with this problem again and I am happy to report I actually seem to have found a solution.

My setup has changed somewhat (now running an FX8300 (with AES-NI) and 16GB as the host for the VM) as has my connection (now 200/50), but I was experiencing the same problem (without OpenVPN, OPNSense managed the full 200/50, with OpenVPN I was stuck at very inconsistent speeds which averaged around 20/20 and maxed out at 30/30 occasionally even after extensive tweaking).

The limiting factor appears to have been the send/receive buffers. I am not sure what default values OpenVPN applies in OPNSense, but I found a thread (https://forum.netgate.com/topic/103216/pfsense-hardware-for-home-router-openvpn-performance) about pfsense discussing OpenVPN performance where someone posted their config including the buffers. Adding them immediately boosted my Download to the expected 200 on download. My download is still stuck around 20-30 but it seems like this is more a limitation of my VPN provider (as I cant get over 30Mbits for upload on my Windows Desktop either)

TL:DR

Add the following lines to the advanced config for OpenVPN connections:

Code Select Expand

sndbuf 524288;
rcvbuf 524288

Maybe this will help someone who read this thread and couldn't get any further than I had come so far :)

In OPNsense, go to Interfaces/Settings. I believe by default, OPNsense has Hardware CRC, Hardware TSO, and Hardware LRO all disabled.

I have not used pfSense in a few years but, I recall they used to leave some of these enabled. Perhaps that could be influencing the results a bit? However since you stated all tests are done within VMs with similar hardware allocated to the VMs, this doesn't fully make sense to me. But, it may be worth checking. Other than that I'm not sure what else it could be.

True, those are all disabled. In both pfSense and OPNsense. I'll check if it has an impact, but in openWRT toggling them did not. It might offload some of the CPU load from the VM to the host.

Edit: They do not appear to make a measurable difference.

I'm often maxing out my upload (~500MB) over OpenVPN if i connect from another 1GB link.

This might help: https://forum.opnsense.org/index.php?topic=6590.0

That's a very interesting thread. I'll have a look at what of that I could apply in my situation tomorrow.
I am not sure I am suffering from the same problem (I do not seem to incur a loss in the quality of the connection, only in the bandwidth) and my speeds are consistent (if slower than expected). But I'm new to this, so I might be way off  8)

I will also give disabling compression a go then as well.

Thanks for the quick and helpful responses :)

PS: Never would have occurred to me to look in the intrusion section for performance improvements  ::)

Edit: Seems that disabling compression is not an option with my VPN provider. They appear to enforce compression. The connection does get established but it does not allow any network access and the log gets filled with the following message:

Code Select Expand

openvpn[22303]: Bad compression stub decompression header byte: 102
I half expected this outcome given that enabling always-on compression is an explicit part of their tutorials and is found in all their OpenVPN configs, but it was still worth a try. They are very responsive in terms of support, so I will send them a message about using no compression or at least adaptive compression. Who knows.

[Intro (skippable)]

Hi folks,

Intro (skippable)
I recently decided to try using a self-hosted router to allow all my network traffic to be routed through my VPN provider transparently. It has been a very interesting journey so far. I started with OpenWRT, but it's stable releases were very old and the snapshots contained too many bugs to use on a daily basis. I then tried pfSense but I kept searching for alternatives and eventually stumbled upon OPNsense and it's vision and style align much better with my preference for software projects.
TL:DR: Recently started using OPNsense

I created essentially identical setups with openWRT, pfSense and OPNsense to tunnel my network traffic through my VPN. I would prefer to continue using OPNsense, however, the performance difference in terms of OpenVPN throughput is staggering:

pfSense (2.4.3) 60Mb/s
openWRT (1.17.04) 85Mb/s
OPNsense (18.1.5) 30Mb/s

For reference on my setup:
All softwares are running in a VirtualBox VM on a Windows 10 Pro host with the following specs:
CPU: Athlon X4 620 @ 3GHz
RAM: 4GB DDR3-1333
Of that, I dedicated 3 cores and 1024MB to the respective VMs and testing was done successively. Network adapters are emulated as Intel PRO/1000 MT Desktop (with the exception of OpenWRT which benefits from paravirtualized network adapters. They are not used on OPNsense and pfSense because in those two, they incur a steep performance penalty). Underlying hardware are Gbit-Realtek NICs (easily capable of pushing more than 100Mb/s consistently).
Connection using direct connection via the provider router:
Down: 100Mb/s (advertised), 90-110Mb/s (actual)
Up: 5Mb/s (advertised), 30Mb/s (actual)
The VPN provider (mullvad.net) uses AES-256-CBC to encrypt the traffic with LZO compression enabled (non-adaptive). They also provide a very complete guide to setup on openWRT and pfSense (which works for OPNsense with essentially no changes).

Now, I am not surprised that my CPU fails to achieve the maximum throughput given the usual performance of OpenVPN/OpenSSL, however, I am very surprised by the performance difference between OPNsense and pfSense. I expected a performance penalty coming from openWRT (given it's designed for embedded systems) but I expected OPNsense to perform similarly to pfSense (if not better).

So to you guys myquestion: is there something obvious in the OPNsense/OpenVPN settings that I might be missing that would massively influence the performance? Or is there a reason I should expect OPNsense to perform much worse in combination with OpenVPN?

Any tips or ideas would be greatly appreciated  :D