PPPOE MSS claimping problem

[Deltorek112]

After updating to 25.7.1 I have problem with accessing https sites.
There were no changes to WAN/PPPOE conection settings, MTU is set to 1492 and MSS to 1456.
Previously it worked fine, now on windows 10 machine I have to set JumboFrames to 1400 to load Gmail.

Also I have backup connection 5G and when I disable PPPOE it works perfectly fine with MTU of 1500 and MSS 1500 (on windows PC JumboFrames are set to 9000)

I tried disabling normalization (Disable scrub interface) and adding rule in detailed settings added rule with max mtu set to 1456, but it did not work.

As for Path MTU Discovery, all ICMP traffic is enabled on all LAN and WAN interfaces in firewall.

Please let me know if some one has and idea how to fix that or need any more details.

[hharry]

It's working for me, i'm running  OPNsense 25.7.1_1-amd64, and confirmed MSS clamping is working.

My LAN PC has MTU = 1500, and a pcaps shows it asks for MSS = 1460 bytes in the tcp syn, and then a pcap run on OPNsense then further confirms the MSS is clamped down to the expected 1452 bytes


Below is my OPNsense config.

[Deltorek112]

Is your config also using PPPoE?
Because it works fine on 5G DHCP connection also for me, but fails on PPPoE

[hharry]

Is your config also using PPPoE?

Sure is, actually i have several PPPoE OPNsense 25.7.1_1 deployments, and all are clearly showing MSS clamping working just fine.

Did you take a look at my screenshot ? you mentioned a MSS 1456 in your original post, which is too large for PPPoE, the largest MSS you'll get, to avoid IP fragmentation is PPPoE MTU/MRU - 40 == 1492 - 40 = 1452....The 40 bytes comes from 20 Bytes for IP header, and 20 bytes of TCP header....the rest is tcp segment payload ( i.e. MSS )

Also you do need to put a valid value in the interface MSS part of the config, if you enter in 1492, OPNsense automatically subtracts the 40 bytes automatically, to give you the perfect MSS 1452 operationally.

[Deltorek112]

Sorry, screenshot was not showing up on phone, set MSS to 1452 and removed MTU and still the same.
Tried ping with size 1466 and it got proper response about it being too big:
ping -f poczta.wp.pl -l 1466
Pinging poczta.wp.pl [193.17.41.249] with 1466 bytes of data:
Packet needs to be fragmented but DF set.

And when I lowered the size to 1464 I got the response:
ping -f poczta.wp.pl -l 1464
Pinging poczta.wp.pl [193.17.41.249] with 1464 bytes of data:
Reply from 193.17.41.249: bytes=1464 time=6ms TTL=60
Reply from 193.17.41.249: bytes=1464 time=5ms TTL=60

All of this was done with MTU empty and MSS set to 1452

But browser is still not able to open Gmail or this site.
Maybe it is a problem with Path MTU Discovery, I run this command:
netsh interface ipv4 show destinationcache
and got this
PMTU Destination Address                           Next Hop Address
1492 193.17.41.249                                 192.168.1.1               <--poczta.wp.pl
1492 142.250.186.197                               192.168.1.1               <--mail.google.com

But the only change was the upgrade of OPNSense, could firewall filter out the protocol needed for this?
I have floating rules to allow all ICMP traffic.(see screenshot)
Do I need to add any more rules to make it work again?

Also without lowering MTU on windows from 9000 to 1400 I could not attach anything to this post was getting 408

[hharry]

TCP MSS only works for TCP protocol, if your using chrome or brave browser, possibly the browser is using http3/quic protocol which is udp based....

I've never tested OPNsense to see if it supports PMTU discovery before, or if it supports IP fragmentation and re-assembly either, but can test etc....

from the cli, from windows cmd prompt, you should try a curl, something like below, which is tcp port 80 connection

curl -vv http://ifconfig.me

Code Select Expand

root@OPNsense:~ # curl -v http://ifconfig.me
* Host ifconfig.me:80 was resolved.
* IPv6: 2600:1901:0:b2bd::
* IPv4: 34.160.111.145
*   Trying [2600:1901:0:b2bd::]:80...
* Immediate connect fail for 2600:1901:0:b2bd::: No route to host
*   Trying 34.160.111.145:80...
* Connected to ifconfig.me (34.160.111.145) port 80
* using HTTP/1.x
> GET / HTTP/1.1
> Host: ifconfig.me
> User-Agent: curl/8.15.0
> Accept: */*
>
* Request completely sent off
< HTTP/1.1 200 OK
< Content-Length: 13
< access-control-allow-origin: *
< content-type: text/plain
< date: Sat, 02 Aug 2025 07:16:36 GMT
< via: 1.1 google
<
* Connection #0 to host ifconfig.me left intact
103.107.16.94root@OPNsense:~ # curl -vv http://ifconfig.me
17:16:44.765112 [0-0] * Host ifconfig.me:80 was resolved.
17:16:44.765273 [0-0] * IPv6: 2600:1901:0:b2bd::
17:16:44.765291 [0-0] * IPv4: 34.160.111.145
17:16:44.765312 [0-0] * [SETUP] added
17:16:44.765345 [0-0] *   Trying [2600:1901:0:b2bd::]:80...
17:16:44.765384 [0-0] * Immediate connect fail for 2600:1901:0:b2bd::: No route to host
17:16:44.765412 [0-0] *   Trying 34.160.111.145:80...
17:16:44.765490 [0-0] * [SETUP] Curl_conn_connect(block=0) -> 0, done=0
17:16:44.766604 [0-0] * [SETUP] Curl_conn_connect(block=0) -> 0, done=0
17:16:44.792995 [0-0] * [SETUP] Curl_conn_connect(block=0) -> 0, done=1
17:16:44.793039 [0-0] * Connected to ifconfig.me (34.160.111.145) port 80
17:16:44.793069 [0-0] * using HTTP/1.x
17:16:44.793149 [0-0] > GET / HTTP/1.1
17:16:44.793149 [0-0] > Host: ifconfig.me
17:16:44.793149 [0-0] > User-Agent: curl/8.15.0
17:16:44.793149 [0-0] > Accept: */*
17:16:44.793149 [0-0] >
17:16:44.793280 [0-0] * Request completely sent off
17:16:45.000968 [0-0] < HTTP/1.1 200 OK
17:16:45.001029 [0-0] < Content-Length: 13
17:16:45.001053 [0-0] < access-control-allow-origin: *
17:16:45.001074 [0-0] < content-type: text/plain
17:16:45.001092 [0-0] < date: Sat, 02 Aug 2025 07:16:44 GMT
17:16:45.001110 [0-0] < via: 1.1 google
17:16:45.001137 [0-0] <
17:16:45.001247 [0-0] * Connection #0 to host ifconfig.me left intact
***reports_your_public IP here***root@OPNsense:~ #

Also, chrome allows you to admin disable http3/quic

To disable HTTP/3 (QUIC) in Chrome, navigate to chrome://flags in the address bar, search for "Experimental QUIC protocol", and set the dropdown to "Disabled". Restart Chrome for the changes to take effect.

[Deltorek112]

After disabling http3/quic nothing changes, chrome is not able to open GMail or the other site(poczta.wp.pl), same on Firefox.
As for curl:

Code Select Expand

curl -vv http://ifconfig.me
09:47:54.658000 [0-0] * Host ifconfig.me:80 was resolved.
09:47:54.661000 [0-0] * IPv6: 2600:1901:0:b2bd::
09:47:54.663000 [0-0] * IPv4: 34.160.111.145
09:47:54.665000 [0-0] * [SETUP] added
09:47:54.667000 [0-0] *   Trying [2600:1901:0:b2bd::]:80...
09:47:54.669000 [0-0] * [SETUP] Curl_conn_connect(block=0) -> 0, done=0
09:47:54.674000 [0-0] * connect to 2600:1901:0:b2bd:: port 80 from :: port 56601 failed: Network unreachable
09:47:54.677000 [0-0] *   Trying 34.160.111.145:80...
09:47:54.679000 [0-0] * [SETUP] Curl_conn_connect(block=0) -> 0, done=0
09:47:54.685000 [0-0] * [SETUP] Curl_conn_connect(block=0) -> 0, done=1
09:47:54.687000 [0-0] * Connected to ifconfig.me (34.160.111.145) port 80
09:47:54.690000 [0-0] * using HTTP/1.x
09:47:54.692000 [0-0] > GET / HTTP/1.1
09:47:54.692000 [0-0] > Host: ifconfig.me
09:47:54.692000 [0-0] > User-Agent: curl/8.13.0
09:47:54.692000 [0-0] > Accept: */*
09:47:54.692000 [0-0] >
09:47:54.701000 [0-0] * Request completely sent off
09:47:54.819000 [0-0] < HTTP/1.1 200 OK
09:47:54.821000 [0-0] < Content-Length: 13
09:47:54.823000 [0-0] < access-control-allow-origin: *
09:47:54.825000 [0-0] < content-type: text/plain
09:47:54.827000 [0-0] < date: Sat, 02 Aug 2025 07:47:54 GMT
09:47:54.829000 [0-0] < via: 1.1 google
09:47:54.831000 [0-0] <
46.205.201.4609:47:54.833000 [0-0] * Connection #0 to host ifconfig.me left intactAnd curl to gmail:

Code Select Expand

curl -vv https://mail.google.com/mail/u/0/
09:47:09.059000 [0-0] * Host mail.google.com:443 was resolved.
09:47:09.061000 [0-0] * IPv6: 2a00:1450:401b:800::2005
09:47:09.064000 [0-0] * IPv4: 216.58.208.197
09:47:09.066000 [0-0] * [HTTPS-CONNECT] adding wanted h2
09:47:09.068000 [0-0] * [HTTPS-CONNECT] added
09:47:09.072000 [0-0] * [HTTPS-CONNECT] connect, init
09:47:09.074000 [0-0] *   Trying [2a00:1450:401b:800::2005]:443...
09:47:09.077000 [0-0] * [HTTPS-CONNECT] connect -> 0, done=0
09:47:09.079000 [0-0] * [HTTPS-CONNECT] Curl_conn_connect(block=0) -> 0, done=0
09:47:09.082000 [0-0] * [HTTPS-CONNECT] adjust_pollset -> 1 socks
09:47:09.084000 [0-0] * connect to 2a00:1450:401b:800::2005 port 443 from :: port 56557 failed: Network unreachable
09:47:09.088000 [0-0] *   Trying 216.58.208.197:443...
09:47:09.090000 [0-0] * [HTTPS-CONNECT] connect -> 0, done=0
09:47:09.093000 [0-0] * [HTTPS-CONNECT] Curl_conn_connect(block=0) -> 0, done=0
09:47:09.095000 [0-0] * [HTTPS-CONNECT] adjust_pollset -> 1 socks
09:47:09.098000 [0-0] * schannel: disabled automatic use of client certificate
09:47:09.101000 [0-0] * ALPN: curl offers http/1.1
09:47:09.103000 [0-0] * [HTTPS-CONNECT] connect -> 0, done=0
09:47:09.106000 [0-0] * [HTTPS-CONNECT] Curl_conn_connect(block=0) -> 0, done=0
09:47:09.108000 [0-0] * [HTTPS-CONNECT] adjust_pollset -> 1 socks
09:47:09.127000 [0-0] * [HTTPS-CONNECT] connect -> 0, done=0
09:47:09.130000 [0-0] * [HTTPS-CONNECT] Curl_conn_connect(block=0) -> 0, done=0
09:47:09.133000 [0-0] * [HTTPS-CONNECT] adjust_pollset -> 1 socks
09:47:09.136000 [0-0] * ALPN: server accepted http/1.1
09:47:09.138000 [0-0] * [HTTPS-CONNECT] connect+handshake h2: 63ms, 1st data: 38ms
09:47:09.141000 [0-0] * [HTTPS-CONNECT] connect -> 0, done=1
09:47:09.143000 [0-0] * [HTTPS-CONNECT] Curl_conn_connect(block=0) -> 0, done=1
09:47:09.146000 [0-0] * Connected to mail.google.com (216.58.208.197) port 443
09:47:09.150000 [0-0] * using HTTP/1.x
09:47:09.152000 [0-0] > GET /mail/u/0/ HTTP/1.1
09:47:09.152000 [0-0] > Host: mail.google.com
09:47:09.152000 [0-0] > User-Agent: curl/8.13.0
09:47:09.152000 [0-0] > Accept: */*
09:47:09.152000 [0-0] >
09:47:09.161000 [0-0] * Request completely sent off
09:47:09.188000 [0-0] < HTTP/1.1 302 Found
09:47:09.190000 [0-0] < Content-Type: application/binary
09:47:09.192000 [0-0] < Vary: Sec-Fetch-Dest, Sec-Fetch-Mode, Sec-Fetch-Site
09:47:09.195000 [0-0] < Location: https://accounts.google.com/ServiceLogin?service=mail&passive=1209600&osid=1&continue=https://mail.google.com/mail/u/0/&followup=https://mail.google.com/mail/u/0/&emr=1
09:47:09.201000 [0-0] < Strict-Transport-Security: max-age=10886400; includeSubDomains
09:47:09.204000 [0-0] < Cross-Origin-Resource-Policy: same-site
09:47:09.206000 [0-0] < Permissions-Policy: ch-ua-arch=*, ch-ua-bitness=*, ch-ua-full-version=*, ch-ua-full-version-list=*, ch-ua-model=*, ch-ua-wow64=*, ch-ua-form-factors=*, ch-ua-platform=*, ch-ua-platform-version=*
09:47:09.212000 [0-0] < Date: Sat, 02 Aug 2025 07:47:09 GMT
09:47:09.215000 [0-0] < Server: ESF
09:47:09.217000 [0-0] < Content-Length: 0
09:47:09.219000 [0-0] < X-XSS-Protection: 0
09:47:09.221000 [0-0] < X-Frame-Options: SAMEORIGIN
09:47:09.223000 [0-0] < X-Content-Type-Options: nosniff
09:47:09.225000 [0-0] < Alt-Svc: clear
09:47:09.228000 [0-0] <
09:47:09.229000 [0-0] * Connection #0 to host mail.google.com left intactAnd to poczta.wp.pl:

Code Select Expand

curl -vv https://poczta.wp.pl/w/mails
09:48:38.651000 [0-0] * Host poczta.wp.pl:443 was resolved.
09:48:38.656000 [0-0] * IPv6: (none)
09:48:38.658000 [0-0] * IPv4: 193.17.41.249
09:48:38.660000 [0-0] * [HTTPS-CONNECT] adding wanted h2
09:48:38.662000 [0-0] * [HTTPS-CONNECT] added
09:48:38.664000 [0-0] * [HTTPS-CONNECT] connect, init
09:48:38.666000 [0-0] *   Trying 193.17.41.249:443...
09:48:38.669000 [0-0] * [HTTPS-CONNECT] connect -> 0, done=0
09:48:38.671000 [0-0] * [HTTPS-CONNECT] Curl_conn_connect(block=0) -> 0, done=0
09:48:38.674000 [0-0] * [HTTPS-CONNECT] adjust_pollset -> 1 socks
09:48:38.676000 [0-0] * schannel: disabled automatic use of client certificate
09:48:38.680000 [0-0] * ALPN: curl offers http/1.1
09:48:38.682000 [0-0] * [HTTPS-CONNECT] connect -> 0, done=0
09:48:38.685000 [0-0] * [HTTPS-CONNECT] Curl_conn_connect(block=0) -> 0, done=0
09:48:38.688000 [0-0] * [HTTPS-CONNECT] adjust_pollset -> 1 socks
09:48:38.690000 [0-0] * [HTTPS-CONNECT] connect -> 0, done=0
09:48:38.693000 [0-0] * [HTTPS-CONNECT] Curl_conn_connect(block=0) -> 0, done=0
09:48:38.695000 [0-0] * [HTTPS-CONNECT] adjust_pollset -> 1 socks
09:48:38.699000 [0-0] * ALPN: server accepted http/1.1
09:48:38.701000 [0-0] * [HTTPS-CONNECT] connect+handshake h2: 34ms, 1st data: 23ms
09:48:38.704000 [0-0] * [HTTPS-CONNECT] connect -> 0, done=1
09:48:38.706000 [0-0] * [HTTPS-CONNECT] Curl_conn_connect(block=0) -> 0, done=1
09:48:38.709000 [0-0] * Connected to poczta.wp.pl (193.17.41.249) port 443
09:48:38.712000 [0-0] * using HTTP/1.x
09:48:38.713000 [0-0] > GET /w/mails HTTP/1.1
09:48:38.713000 [0-0] > Host: poczta.wp.pl
09:48:38.713000 [0-0] > User-Agent: curl/8.13.0
09:48:38.713000 [0-0] > Accept: */*
09:48:38.713000 [0-0] >
09:48:38.723000 [0-0] < HTTP/1.1 302 Moved Temporarily
09:48:38.725000 [0-0] < Server: nginx
09:48:38.727000 [0-0] < Date: Sat, 02 Aug 2025 07:48:38 GMT
09:48:38.729000 [0-0] < Content-Type: text/html
09:48:38.731000 [0-0] < Content-Length: 138
09:48:38.734000 [0-0] < Connection: keep-alive
09:48:38.736000 [0-0] < Location: https://poczta.wp.pl/login/v1/reload
09:48:38.738000 [0-0] < Cache-Control: no-cache
09:48:38.740000 [0-0] < Accept-CH: device-memory, dpr, width, viewport-width, rtt, downlink, ect, sec-ch-ua, sec-ch-ua-platform, sec-ch-ua-mobile, sec-ch-ua-full-version-list, sec-ch-ua-platform-version, sec-ch-ua-arch, sec-ch-ua-bitness, sec-ch-ua-model
09:48:38.747000 [0-0] < Accept-CH-Lifetime: 604800
09:48:38.751000 [0-0] <
<html>
<head><title>302 Found</title></head>
<body>
<center><h1>302 Found</h1></center>
<hr><center>nginx</center>
</body>
</html>
09:48:38.753000 [0-0] * Connection #0 to host poczta.wp.pl left intact

[hharry]

So it seems OPNsesnse supports PMTU discovery, when the original IP datagram exceeds 1492 bytes, and OPNsense sends back the ICMP Destination unreachable (type 3), Fragmentation needed (code 4), with the correct MTU of next hop g/w = 1492 bytes....see below...

[Monviech (Cedrik)]

It does, I do the same for my PPPoE. I use PMTU.

For it to work the firewall must allow ICMP(v4). If its blocked everywhere PMTu cannot work.

[Deltorek112]

So I spun up a ubuntu vm on proxmox to test if linux browser would work, and it does (linux.png).
So something broke for windows 10, or my windows install broke immediately after upgrade of OPNsense, in less then 5 minutes after.

[Deltorek112]

I found a hint

Code Select Expand

ping poczta.wp.pl -M want -s 1500
PING poczta.wp.pl (193.17.41.249) 1500(1528) bytes of data.
From router.ulanow.local.ulanow.local (192.168.1.1) icmp_seq=1 Frag needed and DF set (mtu = 1484)MTU on wan is set to 1492 and no MSS
And with MSS set to 1452 I'm able to do this:

Code Select Expand

ping poczta.wp.pl -M want -s 1456
PING poczta.wp.pl (193.17.41.249) 1456(1484) bytes of data.
1464 bytes from rev-249.go2.pl (193.17.41.249): icmp_seq=1 ttl=60 time=5.78 ms
1464 bytes from rev-249.go2.pl (193.17.41.249): icmp_seq=2 ttl=60 time=5.17 ms
1464 bytes from rev-249.go2.pl (193.17.41.249): icmp_seq=3 ttl=60 time=5.39 ms
^C
--- poczta.wp.pl ping statistics ---
3 packets transmitted, 3 received, 0% packet loss, time 2003ms
rtt min/avg/max/mdev = 5.166/5.445/5.779/0.253 msBut with MTU 1484 and MSS 1444 I get no response for size 1452 but get fragmentation response for 1456:

Code Select Expand

ping poczta.wp.pl -M want -s 1456
PING poczta.wp.pl (193.17.41.249) 1456(1484) bytes of data.
From router.ulanow.local.ulanow.local (192.168.1.1) icmp_seq=1 Frag needed and DF set (mtu = 1476)
^C
--- poczta.wp.pl ping statistics ---
4 packets transmitted, 0 received, +1 errors, 100% packet loss, time 3067ms
root@proxmox:~# ping poczta.wp.pl -M want -s 1452
PING poczta.wp.pl (193.17.41.249) 1452(1480) bytes of data.
^C
--- poczta.wp.pl ping statistics ---
9 packets transmitted, 0 received, 100% packet loss, time 8231ms


[mbk2]

I encountered similar issues with HTTPS timeouts on some websites. Wireshark indicated a successful TLS handshake, but no application data was transmitted afterward. This led me to suspect an MTU/MSS mismatch.

After some research, I found several posts on forums and Reddit mentioning MTU/MSS-related problems with PPPoE connections. I'm not entirely sure whether the issue already existed in 25.1 and I simply didn't notice it, or if it first appeared for me with 25.7.

As a temporary workaround, I manually set the MSS on the LAN interface to a lower value (e.g., 1200), which resolved the issue partially. However, setting the MSS via the "Firewall: Settings: Normalization" feature had no noticeable effect.

On the WAN interface, MTU and MSS were initially unset. Running ifconfig showed a default MTU of 1500 on the physical interface, and 1492 on the PPPoE interface, which is expected. Once I explicitly set the WAN interface MTU to 1492 and MSS to 1452, all connections, including those over VPN tunnels, began working properly again.

When MTU is set to 1492, the PPPoE MTU is automatically reduced to 1484. I ended up setting the WAN MTU back to 1500 and manually defining the MSS as 1452. With this configuration, all services are currently functioning as expected. ifconfig still reports 1500 on the WAN and 1492 on the PPPoE interface, as before.

For context: prior to this, I never explicitly configured MTU or MSS settings, and everything worked out of the box.

[Deltorek112]

Do you also have to use vlan for PPPoE connection?
Also could you share your setup for PPPoE and WAN?
I tried to do the same an issue still persists on windows.

[mbk2]

Yes, vlan for pppoe.

Standard vlan device linked with the point-to-point device. Selected PPPoE in WAN / IPv4 Configuration Type, nothing special. I have dual stack on my wan interface, with IPv6 no issues occoured so far as I know, only with IPv4 server.

Set MTU 1500, MSS 1452 in Interfaces / WAN

Here is my ifconfig with wan interface and pppoe:

Code Select Expand

ix1: flags=1008843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST,LOWER_UP> metric 0 mtu 1500
        options=4803828<VLAN_MTU,JUMBO_MTU,WOL_UCAST,WOL_MCAST,WOL_MAGIC,HWSTATS,MEXTPG>
        ether XX:XX:XX:XX:XX:XX
        media: Ethernet autoselect (1000baseT <full-duplex,rxpause,txpause>)
        status: active
        nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
pppoe0: flags=10088d1<UP,POINTOPOINT,RUNNING,NOARP,SIMPLEX,MULTICAST,LOWER_UP> metric 0 mtu 1492
        description: WAN (wan)
        options=0
        inet XX.XX.XX.XX --> XX.XX.XX.XX netmask 0xffffffff
        inet6 XXXXXXXXXXXXX%pppoe0 prefixlen 64 scopeid 0x1a
        inet6 XXXXXXXXXXXXXXXXXXXXX prefixlen 64 autoconf pltime 1800 vltime 14400
        nd6 options=23<PERFORMNUD,ACCEPT_RTADV,AUTO_LINKLOCAL>


[meyergru]

IDK about changes in new releases, I always solve that problem like this.