Layer 2 encrypted tunnel between two OPNsense boxes

[deajan]

Hello,

I am currently seeking a way to create cheap layer 2 tunnels accros WAN links.
My primarty goal is to interconnect two sites (A and B) just like using a (very long) ethernet cable.

The scenario is the following:
- Site A and site B are in different countries, both have quite okay WAN links (RTT between sites is 14ms)
- Site A has some industrial machines which are operated by some specific industrial computers
- Site B has perfect clones of the industrial computers of site A

Whenever one/more computers of site A fails, I'd like the clones from site B to interact directly with the industrial machines from site A (as disaster plan).

Most of the traffic is layer 3 (TCP/UDP/ICMP), but some traffic is layer 2 (ARP, VLAN, DHCP).
The layer 2 traffic is mandatory for that setup to work, so I am heading for a layer 2 tunnel.

So far I've looked at the following routes:

- L2TP over IPsec: looks like a big overhead to me, ie 128 bytes of headers
- OpenVPN tap: well OpenVPN is very slow compared to IPSec / Wireguard, and I would like to achieve as much bandwidth and low latency as I may get. So OpenVPN is the fallback if I don't get anything to work
- VxLAN (or GENEVE, or GRETAP) over Wireguard: looks promising ?
- Tinc ?
- Zerotier ?

So here are my questions:
What's the best solution (the most performance oriented one, without sacrificing security) ?
Has anyone achieved a good performing layer 2 tunnel setup with OPNsense yet ?

Any feedback is appreciated.

Thanks.

PS: This is my first post on the OPNsense forum (former pfSense user), so I probably don't know OPNsense good enough yet

[mimugmail]

I'd go for OpenVPN bridging as it is the most supported solution.
Tinc and ZT are way slower than OpenVPN.

Never tried VXLAN via WG but it could work ..

[deajan]

Thank you for that answer.

OpenVPN isn't really the performance choice if I remember my past experiences with it (didn't get better than 30% of my raw bandwidth most of the times).

I used to try to speedup openvpn by using UDP encapsulation, aes-ni cpu support, tcp offloading and tunnel jumbo MTU sizes.
Are these options possible with OPNsense ?

Best regards.

[mimugmail]

You can run WAN links with jumbo frames???

[deajan]

AFAIK you can't have jumbo frames on WAN links

But the --tun-mtu option is for the internal tunnel only, see https://community.openvpn.net/openvpn/wiki/Gigabit_Networks_Linux (tweaked setup part).

To be honest, of the 4 tuning options I talked asked about, I never experienced this particular one myself, I just added it to my wishlist.

[deajan]

Has anyone used OPNsense with VXLAN/GENEVE/GRETAP over Wireguard successfully ?

[skydiablo]

so i'm also intrested in this challamge. i my opinion the real challange is to set the MTU in an right size.

starting from an PPPoE connection over the wireguard tunnel throught the VXLAN. and than bridge this vxlan via bridge to an outside interface. so all of this interfaces have an different MTU value. is there an real knowing hacker out there that can calc all this values and bring the stack running?

volker.