Jumbo frames on axgbe / DEC7x0

[newsense]

With FreeBSD 14.3beta2 out, if this gets an actual fix I'd be surprised if it makes it prior to 14.4 upstream.

( With OPNsense that would most likely be included in the next dot release that has a kernel update and at least one test kernel before that )

[meyergru]

Much less so if nobody opens an issue on FreeBSD, where it belongs.

[patient0]

Out of curiosity I restarted my DEC740 with VyOS (on a stick) and in that setup I was able to set the MTU to 9000 (eth3 below) and ping was successful. No hardware restriction, software only.

The setup was: DEC740 (eth3, 10.66.6.1) - MikroTik Switch CSS610-8P-2S+IN - QNAP QNA-UC5G1T (5Gbit USB NIC, 10.66.6.2)

Code Select Expand

vyos@vyos# run sh inter ethernet eth3
eth3: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 9000 qdisc mq state UP group default qlen 1000
    link/ether f4:90:ea:00:73:63 brd ff:ff:ff:ff:ff:ff
    altname enp6s0f1
    inet 10.66.6.1/24 brd 10.66.6.255 scope global eth3
       valid_lft forever preferred_lft forever
    inet6 fe80::f690:eaff:fe00:7363/64 scope link
       valid_lft forever preferred_lft forever

    RX:   bytes  packets  errors  dropped  overrun       mcast
         226837      366       0        0        0         109
    TX:   bytes  packets  errors  dropped  carrier  collisions
         160576       88       0        0        0           0

vyos@vyos# ping -M do -s 8972 -c 4 10.66.6.2
PING 10.66.6.2 (10.66.6.2) 8972(9000) bytes of data.
8980 bytes from 10.66.6.2: icmp_seq=1 ttl=64 time=1.13 ms
8980 bytes from 10.66.6.2: icmp_seq=2 ttl=64 time=1.29 ms
8980 bytes from 10.66.6.2: icmp_seq=3 ttl=64 time=1.42 ms
8980 bytes from 10.66.6.2: icmp_seq=4 ttl=64 time=1.23 ms
Trying the same with FreeBSD 15-CURRENT 2025-05-08 showed the same issue as with OPNsense.

Btw: Linux also reports 3 rx and 3 tx queues, seems to be implemented that way in hardware.

[Kaya]

Thanks for your definitive validation of the hardware capabilities @patient0. Since I only have access to one device, which is in active use as a firewall, I can't do such experiments.

[patient0]

Thanks for your definitive validation of the hardware capabilities @patient0. Since I only have access to one device, which is in active use as a firewall, I can't do such experiments.

No worries, I do like trying stuff like that out.
And while I only have one DEC740 I do have multiple replacements since my home network is pretty simple and there's nobody to complain when I take it down :).

[newsense]

A test kernel has been provided on Github by Stephan

Code Select Expand

opnsense-update -zkr 25.1.6-axgbe

[Kaya]

Yes, that is great news. I will give the patched kernel a try in the next couple of days.

I am running OPNsense Business 25.4. I assume that the patched kernel version (which obviously refers to 25.1) should not create any hiccups because of different OPNsense versions, right?

[newsense]

No should be all good this time. In case of an issue booting the older kernel will get you back to 25.4.

Snapshots are also an option but may be overkill for this particular issue.

[Kaya]

For those, who are interested, but not following the GitHub issue:

I installed the patched kernel, rebooted, changed MTU to 9000 and performed a ping test again. Initially, results looked great, ping was working with a payload size of 8192 bytes. But after some more experiments, the OPNsense crashed and rebooted. Since the box was rock solid before, I assume that the crash is related to the large MTU.

Therefore, for the time being, I reverted the MTU to 1500 bytes.