Hi everyone,
I am having a performance issue with a WireGuard tunnel between my home OPNsense firewall and a VPS hosted at HostBrr.
Setup:
- Home side: OPNsense as WireGuard server
- Remote side: FreeBSD VPS at HostBrr as WireGuard client
- WireGuard tunnel network: 10.10.10.0/24
- VPS WireGuard IP: 10.10.10.2
- Home LAN: 192.168.3.0/24
- iperf3 target inside the home LAN: 192.168.3.2
- WireGuard MTU is currently set to 1420, according to the guide I followed
The WireGuard tunnel itself is up and working. Routing also works; the VPS can reach the LAN host 192.168.3.2.
Issue:
The performance is highly asymmetric.
From the VPS to the LAN host, I get around 160 Mbit/s:
iperf3 -c 192.168.3.2 -P 4
[SUM] 0.00-10.05 sec 205 MBytes 171 Mbits/sec 305 sender
[SUM] 0.00-10.05 sec 196 MBytes 163 Mbits/sec receiver
In the reverse direction, using -R, I only get around 16 Mbit/s:
iperf3 -c 192.168.3.2 -P 4 -R
Reverse mode, remote host 192.168.3.2 is sending
[SUM] 0.00-10.02 sec 23.2 MBytes 19.5 Mbits/sec 2828 sender
[SUM] 0.00-10.01 sec 19.4 MBytes 16.2 Mbits/sec receiver
The very high retransmit count stands out:
Retr: 2828
There are also several intervals in the reverse test showing 0.00 Bytes, so TCP seems to stall completely for short periods.
What I suspect:
I am not sure whether this is caused by one of the following:
- MTU/MSS issue in the WireGuard tunnel, even though MTU is currently 1420
- Missing or incorrect TCP MSS clamping on OPNsense
- Upload limit or packet loss on the home connection
- VPS/provider issue at HostBRR
- Firewall/NAT rule issue on OPNsense
- CPU limitation on either OPNsense or the VPS
Questions:
- Is 1420 a reasonable MTU for this setup, or should I still test lower values such as 1380 or 1360?
- Should I enable TCP MSS clamping on OPNsense? If yes, on which interface/rule and with which value?
- Where should I check on OPNsense for packet drops, state issues, or packet loss?
- Are there any known issues or best practices when using OPNsense as the WireGuard server and FreeBSD as the client?
- Could this kind of asymmetry be caused by the home upload link, even though the retransmit count is so high?
Any hints on what I should check on the OPNsense side would be appreciated.
Thanks!