Messages

[MTU = 1392]

...
For me
MTU = 1392
seems to work fine.

Much appreciated suggestion for a solution. That might just be the cause for what I experienced too.
I'm not quite sure, but I may have just blanked out and not even tried adjusting any MTUs and or MSSs values.

Can't test right now, I've long since reverted back to running multiple OpenVPN gateways instead because I also experienced some janky behavior having multiple WireGuard-VPN clients balanced balanced gateways to WAN.

If this solves handshake, and I if i can also fix the janky balancing of WG in OpenSense this would spare me from A LOT of needless complexity and a great deal of hardware resources.

Due to this HTTPS-issue and a few other issues my current plan of action (when my schedule allowed) was to just break out VPN completely from the OpenSenses and set up multiple balanced virtual WG-gateways in DMZ,
load balance LANs through WG-Hosts, the have WG-hosts failover the OpnSense-boxes, each WG-host having a dedicated path. A real bastard solution for a hobbyist with 2x or more Hosts and 4x IFs :S

Many thanks.
If I get around to setting up a new Test-environment (old one is in the grave) I'll report back if this was the solve for me too.

[Working]

I use OpnSense as a VPN-Gateway for all outgoing traffic to the internet for LAN clients.
I've just recently switched to a single Wireguard Peer as Gateway (previously two OpenVPN-clients as load balanced gateways).

I run OpnSense on bare metal. Supermicro A1SAi, Intel Atom2750F with 4 i354 Nics.

In general everything works great with Wireguard, great latencies, great speeds (>400Mbit/s at 25% load over all cores /w kmod).
But I'm having trouble with a specific website not working over the VPN tunnel.

At first i though '-Hey maybe the exit-node is blocked, in some DDos-filter or something', but that does not seem to be the case here.

The site in question works perfectly without VPN in OpnSense and also with OpenVPN in OpnSense.
the site also works perfectly using the exact same Wireguard-peer connection directly on from a LAN-machine tunneling directly from the client through OpnSense (exact same exit node IP).
But the site can not be loaded if OpnSense is the responsible for running the Wireguard tunnel.

When i run a TCP-Dump it seems that i can actually communicate with the web-server, but I'm missing a specific packet in the TCP-stream when the traffic.
Even with Wireguard up in OpnSense I can telnet to port 80 on the site and get a HTTP-reply back with an actual HTML-response. But I can't establish a proper TLS-session by any means in any browser or using  curl/PowerShell etc.

In the test scenarios with packet dump:
The LAN-client use the same DNS-servers
Underlying IP in the A-record is the same
Wireguard exit node IP is the same
Both machines can reach the server and do get replies back on a packet level.
With OpnSense running Wireguard, the first packet in the Server Hello gets lost on the way to the client,
and is missing from the RAW TCP-dump (All WAN+WG+LAN IFs dumped in OpnSense)

I've tried both with Wireguard-Go and with KMOD, but the result is the exact same.

The stream working vs. broken looks like this (color marked green/red for the packet missing)
 
Working (Wireguard from Lan Client)
TLS   > Client Hello
TCP   < ACK
TLS < Server Hello
TLS   < PSH, ACK (Continuation Data)
TLS   < PSH, ACK (Continuation Data)
TCP   > ACK
TLS   < Certificate Verify finished
And then rest of the handshake and web-data follows just fine

Broken (Wireguard in OpnSense)
TLS   > Client Hello
TCP   < [ACK]
Here the first packet of the TLS Server Hello is always missing in TCP stream
TLS   < [TCP Previous fragment not captured], Continuation Data But I still get continuation data for the missing Server Hello
TCP   > [TCP Dup ACK]
TLS < Continuation Data
TCP   > [TCP Dup ACK]
TCP   > [Keep-Alive]
TLS   < [Keep-Alive, Ack]
And here its just a long string of trying again, trying again, trying again until timeout.

I've Been trying to solve this for a a few days but I can't come up with any reasonable next step in troubleshooting this.
Any bright ideas would be greatly appreciated!

I didn't try the patch, but I did update to 19.7.3 and it's working better now.

Thanks!

Hi!

My problem started after upgrading from 19.7.1 to 19.7.2.

I run multiple (4-6) load balanced OpenVPN Clients from my OpenSense to commercial VPN-Services,
all the VPN-Client have been set to use one of multiple WAN-IF as outgoing Interface.
All outgoing user traffic is set to use only gateway group with tunnel gateways.

As soon as i have any one client-tunnel established, all consecutive attempts to bring another OpenVPN Client up will fail.

I looks as if OpenSense is ignoring interface-selection in the configuration and is instead trying route through the already established client-tunnel over the active TUN-IF (OVPNC#) instead of WAN-IF.

Any one of my OpenVPN ClientConfigs will work as long as it is the first to activate.
On rare occasion i can get two of the Clients up if they are both activated in very rapid succession,
but that's not a very common occurrence (race condition!?).

I've since done a few attempts downgraded back to 19.7.1 everything works fine,and upgraded to 19.7.2 and then it breaks again.
MultiWAN and 4-6 VPN Clients with Load balancing for has been working well for me on prior 19.x.x versions, except having to fiddle a bit when 19.7 hit (the common problem with broken gateways).

Am i missing some new setting in 19.7.2? or is this a bug?
I haven't seen anyone else complain about this over the past few weeks,
Maybe this just an uncommon setup...

BR