GRE over NAT

Started by d4rkd3n1337, August 25, 2024, 11:07:05 AM

Previous topic - Next topic
Hello, guys. I really hope that there are experts among you.
I have next setup:
First site:
OPNsense edge gate\fw (ISP public ip, for example 1.1.1.1) (DMZ ip: for example 10.1.1.1)
Cisco Router in DMZ with tunnels (GRE) interfaces: 10.1.1.2

Second site:
OPNsense edge gate\fw (ISP public ip, for example: 2.2.2.2)

I have working GRE tunnel by scheme:
S2 OPNSense -> MyOPNSense ->  (NAT GRE) -> Cisco
by this scheme I have ~60-80mbit troughpout
Today, for testing I made GRE tunnel in local network (vm to cisco), and I get over 600mbit!
Maybe, in OPNsense have settings for GRE over NAT? Because it's very strange.
What can bottleneck?

Ex configs:
Cisco:
interface ga0/0 - ip address 10.1.1.2/24 (DMZ)
interface Tunnel2
(GRE) ip address 10.0.91.1/30
(GRE) tunnel source 10.1.1.2
(GRE) tunnel destination 2.2.2.2

S2 OPNsense:
em0 (WAN, public ISP, anyway...) - ip: 2.2.2.2
gre0:
source - 2.2.2.2
destination - 1.1.1.1
gre local 10.0.91.2/30

If need, I can provide more info
And sorry for my bad english

August 25, 2024, 11:27:50 AM #1 Last Edit: August 25, 2024, 11:30:50 AM by doktornotor
I'd say you need some MTU/MSS clamping. Firewall ‣ Settings ‣ Normalization. I'd start with something like 1360 for the Max MSS.


Good idea. But, it is globally, right? On GRE sites (cisco\opnsense mtu set to 1476)
From here, I have next question - where natting GRE, we decrement MTU?

August 25, 2024, 11:37:10 AM #3 Last Edit: August 25, 2024, 11:38:54 AM by doktornotor
No, it is not globally, it's per interface (group). See the hints here: https://docs.opnsense.org/manual/how-tos/wireguard-s2s.html


so, I'm trying start iperf with udp, (iperf3 -c server.behind.tunnel -u -b 120M) and get 120Mbit\s (of course with losses, its UDP). But I don't understand yet, where do I need to set MSS? In the firewall settings (normalization) or on the gates with GRE? or both (OPNsense with GRE, OPNsense with NAT GRE and Cisco with GRE)?

What's unclear about the normalization settings from the link I've posted? (It's about Wireguard but it's exactly the same place and same settings -- just applied to GRE interface(s).