LCP negotiation with MRU option in PPPoE, and MTU setting for OPNsense

[camellia]

My ISP uses a PPPoE line with separate PPPoE sessions for IPv4 and IPv6. Since the username for IPv4 and the username for IPv6 are different, IPv4 and IPv6 cannot be shared in a single PPPoE session.

Configuration for IPv4 is standard "IPv4 over PPPoE", and configuration for IPv6 is "DHCPv6-PD over PPPoE". Both IPv4 and IPv6 work fine.

MRU of PPPoE server is 1454. PPPoE server performs "LCP negotiation" with MRU=1454 option, and OPNsense accepts MRU=1454. However, OPNsense sets MTU to 1492 instead of 1454. Normally, if OPNsense accepts MRU=1454 of PPPoE server in LCP negotiation, OPNsense should set MTU to 1454 or less, but this is not actually the case.


I checked the MTU with the following combinations (a) to (f).

The former value is the MTU after Bootup, and the latter value is the MTU after reload. Reload means clicking the "Reload icon" in the Commands column of the Interfaces Overview.

[One PPPoE session]
After bootup, both IPv4 and IPv6 have MTU=1492. After reload, IPv4 has MTU=1492.

(a) WAN  - pppoe0 - IPv4: 1492 / 1492

(b) WAN  - pppoe0 - IPv6: 1492 / N/A

[Two PPPoE sessions]
After bootup, both IPv4 and IPv6 have MTU=1492. After reload, IPv4 has MTU=1492 and IPv6 has MTU=1454.
For IPv4 and IPv6 combinations, when I check the logs, the PPPoE process always occurs in the order IPv4 first and IPv6 second.

(c) WAN  - pppoe0 - IPv4: 1492 / 1492
    OPT1 - pppoe1 - IPv6: 1492 / 1454

(d) WAN  - pppoe1 - IPv6: 1492 / 1454
    OPT1 - pppoe0 - IPv4: 1492 / 1492

(e) WAN  - pppoe1 - IPv4: 1492 / 1492
    OPT1 - pppoe0 - IPv6: 1492 / 1454

(f) WAN  - pppoe0 - IPv6: 1492 / 1454
    OPT1 - pppoe1 - IPv4: 1492 / 1492


From the results of (a) to (f), it appears that after reload, the first PPPoE process will result in MTU=1492, and the second PPPoE process will result in MTU=1454.

I check MTU on OPNsense with:

• ifconfig output
• Interface Overview Details
• Routes Status

I set MTU to blank in Generic configuration of Interfaces for OPNsense.

Below is the LCP negotiation log:

[wan_link0]   MAGICNUM 0x5848d693
[wan_link0]   AUTHPROTO CHAP MD5
[wan_link0]   MRU 1454
[wan_link0] LCP: SendConfigAck #1
[wan_link0]   MAGICNUM 0x5848d693
[wan_link0]   AUTHPROTO CHAP MD5
[wan_link0]   MRU 1454
[wan_link0] LCP: rec'd Configure Request #1 (Req-Sent)

[opt1_link0]   MAGICNUM 0x43d4f057
[opt1_link0]   AUTHPROTO CHAP MD5
[opt1_link0]   MRU 1454
[opt1_link0] LCP: SendConfigAck #1
[opt1_link0]   MAGICNUM 0x43d4f057
[opt1_link0]   AUTHPROTO CHAP MD5
[opt1_link0]   MRU 1454
[opt1_link0] LCP: rec'd Configure Request #1 (Req-Sent)

[camellia]

Today introduces a change in MTU handling for parent interfaces mostly noticed by PPPoE use where the respective MTU values need to fit the parent plus the additional header of the VLAN or PPPoE.  Should the MTU already be misconfigured to a smaller value it will be used as configured so check your configuration and clear the MTU value if you want the system to decide about the effective parent MTU size.

I found the above description of the parent interfaces MTU handling changes related to PPPoE in the OPNsense 23.7.5 release announcement.

Is the reason why the MTU is set to 1492 because the manual MTU setting in the Generic configuration takes precedence over the automatic MTU setting by LCP negotiation? (When the MTU setting is blank, "Calculated PPP MTU: 1492" will be displayed.)

[camellia]

I tested with an older OPNsense and confirmed that OPNsense 23.7.2 sets the MTU to 1454, and that OPNsense 23.7.3 sets the MTU to 1492.

Updating from OPNsense 23.7.2 to OPNsense 23.7.3 did not require a reboot, and the MTU was 1454 after the update. After that, the MTU was 1492 after PPPoE session reconnection the by reload or reboot.

This regression may be due to the "interfaces: further improve PPP MTU handling" in the OPNsense 23.7.3 release announcement.

[franco]

The commit in question is likely https://github.com/opnsense/core/commit/8c9c56f9b504f992fd

But I'm not sure where the issue is since you haven't published your MTU settings (both interface and PPP device).

If MRU is accepted but not set I'm not sure we can do much as we never see it and MPD5 should likely enforce it.


Cheers,
Franco

[camellia]

But I'm not sure where the issue is since you haven't published your MTU settings (both interface and PPP device).

MTU settings for both interface and PPP device are blank.

MPD5 version for both OPNsense 23.7.2 and OPNsense 23.7.3 is 5.9_16. Also, MPD5 has not been reinstalled on OPNsense 23.7.3. However, results of automatic MTU setting are clearly different between OPNsense 23.7.2 and OPNsense 23.7.3.

Since the results of the automatic MTU setting are different even though there is no change in MPD5, I don't think it's an MPD5 issue.

I noticed that the behavior of MTU settings for PPP devices is significantly different between OPNsense 23.7.2 and OPNsense 23.7.3.

[OPNsense23.7.2]
MTU setting for PPP devices in OPNsense 23.7.2 respects the result of LCP negotiation.

• If I set a value larger than 1454, that is not reflected in the actual MTU.
• If I set a value smaller than 1454, that is reflected in the actual MTU.
• If I set it to 1492, the actual MTU remains at 1454.

[OPNsense23.7.3]
MTU setting for PPP devices in OPNsense 23.7.3 ignores the result of LCP negotiation.

• If I set a value larger than 1454, that is reflected in the actual MTU.
• If I set a value smaller than 1454, that is reflected in the actual MTU.
• If I set it to 1492, the actual MTU is 1492.

OPNsense may be implemented in such a way that if the MTU setting is blank, it assumes that 1492 is set. If this assumption is correct, the results of the automatic MTU setting in OPNsense 23.7.2 and OPNsense 23.7.3 will be consistent with the expected MTU.

I am concerned about the statement in the MTU setting help that says "MTU will default to 1492".

[hharry]

i can confirm the OP's point here, i as able to re-create the issue in LAB, can clearly see the received PPPoE servers MRU = 1472, however OPNsense ignores the PPPoE servers MRU, and OPNsense just defaults to a PPPoE interface MTU = 1492


Below can clearly see the PPPoE servers LCP advertised MRU = 1472, which OPNsesne LCP acks.

PPPoE server MAC address in below tcpdump == 00:0c:29:2a:de:ad

OPNsense PPPoE client interface MAC address == 00:0c:29:55:0d:8a

Code Select Expand

root@OPNsense_LAB:~ # tcpdump -nevi vmx0
tcpdump: listening on vmx0, link-type EN10MB (Ethernet), snapshot length 262144 bytes
08:05:48.180841 00:0c:29:55:0d:8a > ff:ff:ff:ff:ff:ff, ethertype PPPoE D (0x8863), length 36: PPPoE PADI [Host-Uniq 0x40517C3B00F8FFFF] [Service-Name]
08:05:48.201268 00:0c:29:2a:de:ad > 00:0c:29:55:0d:8a, ethertype PPPoE D (0x8863), length 74: PPPoE PADO [AC-Name "VYOS03"] [Service-Name] [AC-Cookie 0x7A9041E0E45617BFDABF9F34EF35229B4B7E454C99D2D63C] [Host-Uniq 0x40517C3B00F8FFFF]
08:05:48.201293 00:0c:29:55:0d:8a > 00:0c:29:2a:de:ad, ethertype PPPoE D (0x8863), length 74: PPPoE PADR [Host-Uniq 0x40517C3B00F8FFFF] [AC-Cookie 0x7A9041E0E45617BFDABF9F34EF35229B4B7E454C99D2D63C] [AC-Name "VYOS03"] [Service-Name]
08:05:48.221657 00:0c:29:2a:de:ad > 00:0c:29:55:0d:8a, ethertype PPPoE D (0x8863), length 60: PPPoE PADS [ses 0x180] [AC-Name "VYOS03"] [Service-Name] [Host-Uniq 0x40517C3B00F8FFFF]
08:05:48.221879 00:0c:29:55:0d:8a > 00:0c:29:2a:de:ad, ethertype PPPoE S (0x8864), length 36: PPPoE  [ses 0x180] LCP (0xc021), length 16: LCP, Conf-Request (0x01), id 1, length 16
        encoded length 14 (=Option(s) length 10)
          MRU Option (0x01), length 4: 1492
          Magic-Num Option (0x05), length 6: 0xcb840b5a
08:05:48.221932 00:0c:29:2a:de:ad > 00:0c:29:55:0d:8a, ethertype PPPoE S (0x8864), length 60: PPPoE  [ses 0x180] LCP (0xc021), length 21: LCP, Conf-Request (0x01), id 137, length 21
        encoded length 19 (=Option(s) length 15)
          Auth-Prot Option (0x03), length 5: CHAP, MD5
          MRU Option (0x01), length 4: 1472
          Magic-Num Option (0x05), length 6: 0x5a0802f3
08:05:48.222042 00:0c:29:55:0d:8a > 00:0c:29:2a:de:ad, ethertype PPPoE S (0x8864), length 41: PPPoE  [ses 0x180] LCP (0xc021), length 21: LCP, Conf-Ack (0x02), id 137, length 21
        encoded length 19 (=Option(s) length 15)
          Auth-Prot Option (0x03), length 5: CHAP, MD5
          MRU Option (0x01), length 4: 1472
          Magic-Num Option (0x05), length 6: 0x5a0802f3
08:05:48.242128 00:0c:29:2a:de:ad > 00:0c:29:55:0d:8a, ethertype PPPoE S (0x8864), length 60: PPPoE  [ses 0x180] LCP (0xc021), length 16: LCP, Conf-Ack (0x02), id 1, length 16
        encoded length 14 (=Option(s) length 10)
          MRU Option (0x01), length 4: 1492
          Magic-Num Option (0x05), length 6: 0xcb840b5a



Operationally the PPPoE interface MTU on OPNsense still defaults to 1492, despite LCP ack'ing the remote PPPoE servers MRU = 1472

Code Select Expand

root@OPNsense_LAB:~ # ifconfig
pppoe1: flags=10088d1<UP,POINTOPOINT,RUNNING,NOARP,SIMPLEX,MULTICAST,LOWER_UP> metric 0 mtu 1492
        description: WAN (wan)
        options=0
        inet 192.168.40.1 --> 192.168.40.254 netmask 0xffffffff
        nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>


OPNsense not handling any vlan's in above example!, all vlan handling performed in ESXi vSwitch port groups...

Most vendors consider such behavior as a clear and repeatable defect, needing fixing !