Hi,
Two SIP phones have been working fine using NAT Port Forward for more than a year now, but some of the changes implemented from opnsense-24.7.1 onward have broken part of the VoIP connection.
Both phones contact the provider's SIP gateway and register/re-register. Incoming calls can be received and answered. The problem is experienced with placing outgoing calls. They fail to connect and soon time out returning a network failure on the phone display.
Looking at packet captures on WAN and LAN the phones send out repeated INVITE packets when trying to set up an outbound call, but they do not receive a response from the SIP gateway.
Re-installing opnsense 24.7 restores the full VoIP functionality.
My network topology:
Phone 1
=======
IP address: 192.168.20.21
Configured SIP port: 5062
Configured RTP ports: 49252 - 49262
Phone 2
=======
IP address: 192.168.20.22
Configured SIP port: 5064
Configured RTP ports: 49272 - 49282
OPNSense
========
WAN IP address: 123.45.6.789
I have set up the ISP's SIP gateway IP ranges as aliases under SIP_IPv4 & RTP_IPv4.
These are my Port Forward rules:
Source Destination NAT
Interface Proto Address Ports Address Ports IP Ports Description
LAN TCP * * LAN address 22, 80, 443 * * Anti-Lockout Rule
WAN UDP SIP_IPv4 5060 (SIP) 123.45.6.789 * 192.168.20.21 5062 SIP IPv4 redirect
WAN UDP SIP_IPv4 5060 (SIP) 123.45.6.789 * 192.168.20.22 5064 SIP IPv4 redirect
WAN UDP RTP_IPv4 * 123.45.6.789 49252 - 49262 192.168.20.21 49252 - 49262 RTP IPv4 Redirect
WAN UDP RTP_IPv4 * 123.45.6.789 49272 - 49282 192.168.20.22 49272 - 49282 RTP IPv4 Redirect
Looking at the console output I can see the SIP port forward rule for 192.168.20.22 is missing - not sure why:
~ # pfctl -s nat
[snip ...]
rdr on pppoe0 inet proto udp from <SIP_IPv4> port = sip to 123.45.6.789 -> 192.168.20.21 port 5062
rdr on pppoe0 inet proto udp from <RTP_IPv4> to 123.45.6.789 port 49252:49262 -> 192.168.20.21 port 49252:49262
rdr on pppoe0 inet proto udp from <RTP_IPv4> to 123.45.6.789 port 49272:49282 -> 192.168.20.22 port 49272:49282
I'm a bit at a loss here as to what is amiss. Do I need to change the Port Forward settings to be able to restore full VoIP functionality with the latest opnsense 24.7.3_1 kernel changes?
How could you forward the WAN port 5060 to both IP phones at once? I would guess first one wins.
I think that you must not have port translation. For the RTP ports, this is mandatory, since the SIP signalisation done by the phones tells the registrar on which ports the RTP connection must be done. If the firewall interferes with that via port translation, it would not work.
So, just like with the RTP ports, you would have to use ports 5062 and 5064 on the WAN interface as well.
Also, I would not limit the rule to some known IPs of the SIP registrar, as they could change in the future. If you want to secure the ports, use the ASN of the registrar.
Thank you meyergru,
Quote from: meyergru on September 01, 2024, 05:43:07 PMI think that you must not have port translation. For the RTP ports, this is mandatory, since the SIP signalisation done by the phones tells the registrar on which ports the RTP connection must be done. If the firewall interferes with that via port translation, it would not work.
OK.
Quote from: meyergru on September 01, 2024, 05:43:07 PMSo, just like with the RTP ports, you would have to use ports 5062 and 5064 on the WAN interface as well.
I have now set up port forwarding from WAN:5062 to one phone and from WAN:5064 to the other phone. It certainly looks better:
rdr on pppoe0 inet proto udp from <SIP_IPv4> port = sip to 123.45.6.789 port = 5062 -> 192.168.20.21 port 5062
rdr on pppoe0 inet proto udp from <SIP_IPv4> port = sip to 123.45.6.789 port = 5064 -> 192.168.20.22 port 5064
rdr on pppoe0 inet proto udp from <RTP_IPv4> to 123.45.6.789 port 49252:49262 -> 192.168.20.21 port 49252:49262
rdr on pppoe0 inet proto udp from <RTP_IPv4> to 123.45.6.789 port 49272:49282 -> 192.168.20.22 port 49272:49282I also rebooted the phones so they re-register with the ISP. It hasn't worked. I still cannot place outgoing calls. :(
Quote from: meyergru on September 01, 2024, 05:43:07 PMAlso, I would not limit the rule to some known IPs of the SIP registrar, as they could change in the future. If you want to secure the ports, use the ASN of the registrar.
Good point, thanks.
I'm not sure how else to troubleshoot this problem. Is there something else I could try on the opnsense configuration, or should I just sit this out and stay on 24.7 for now? If my opnsense configuration is OK, then it must be something related to the recent kernel changes. :-\
You did use "static port" for the connectiions?
I've had a similar issue with Verizon and outgoing calls over "wifi calling." I have yet to find any setting I can change to make it work, but this has persisted for a very long time and isn't new in 24.7 for me. Responding to see what the resolution is here to see if it works for my issue as well!
Quote from: meyergru on September 01, 2024, 11:35:56 PM
You did use "static port" for the connectiions?
Sorry, I am not familiar with many opnsense configuration options - are you referring to the Outbound NAT option to enable a "static port"? I have not changed the Outbound NAT from 'auto'. I only have one IPv4 public address at this moment.
Also, the SIP registrar responds to the correct ports for each phone, each phone registers successfully and incoming calls can be answered. It is the outgoing calls which fail.
Quote from: SideOfRanch on September 02, 2024, 12:31:04 AM
I've had a similar issue with Verizon and outgoing calls over "wifi calling." I have yet to find any setting I can change to make it work, but this has persisted for a very long time and isn't new in 24.7 for me.
Just to make it clear, the 24.7 and previous releases work OK in my use case, it's versions after 24.7 which partially broke VoIP here. Are you able to receive calls with "wifi calling", or does it not work at all?
Quote from: fbantgat7 on September 02, 2024, 10:46:29 AM
Quote from: meyergru on September 01, 2024, 11:35:56 PM
You did use "static port" for the connectiions?
Sorry, I am not familiar with many opnsense configuration options - are you referring to the Outbound NAT option to enable a "static port"? I have not changed the Outbound NAT from 'auto'. I only have one IPv4 public address at this moment.
Also, the SIP registrar responds to the correct ports for each phone, each phone registers successfully and incoming calls can be answered. It is the outgoing calls which fail.
Of course, it cannot work like that.
Set the outbound NAT mode to 'Hybrid outbound NAT rule generation', then add a new rule like so...
Interface: WAN
TCP/IP Version: IPv4
Source address: IP of VOIP Device
Source port: udp/ *
Destination address: any
Destination port: any
Translation/target: Interface address
Static Port: yes
NAT will translate ports to use a random port, your VoIP provider will try to respond on this random port and fail. The static outbound port, ensures that the outbound port number is not changed. SIP expects the source and destination ports to be the same.
Thanks meyergru,
I added two Outbound rules one for each phone. I now have:nat on pppoe0 inet proto udp from 192.168.20.21 to <SIP_IPv4> -> (pppoe0:0) static-port
nat on pppoe0 inet proto udp from 192.168.20.22 to <SIP_IPv4> -> (pppoe0:0) static-port
[snip ...]
rdr on pppoe0 inet proto udp from <SIP_IPv4> port = sip to 123.45.6.789 port = 5062 -> 192.168.20.21 port 5062
rdr on pppoe0 inet proto udp from <SIP_IPv4> port = sip to 123.45.6.789 port = 5064 -> 192.168.20.22 port 5064
rdr on pppoe0 inet proto udp from <RTP_IPv4> to 123.45.6.789 port 49252:49262 -> 192.168.20.21 port 49252:49262
rdr on pppoe0 inet proto udp from <RTP_IPv4> to 123.45.6.789 port 49272:49282 -> 192.168.20.22 port 49272:49282
On 24.7 VoIP continues to work as expected. I now upgraded to 24.7.3_1 and again I am not able to place an outbound call.
I wonder if I should just move these phones to IPv6 only, to do away with NAT problems. TBH I have been avoiding doing it because IPv6 on opnsense has been a bit hit & miss over the years.
Is there anything else I should try configuring on 24.7.3_1 to solve this problem?
I use SIP and have no problems with 24.7.x.
But wait a minute - what is this:
Source Destination NAT
Interface Proto Address Ports Address Ports IP Ports Description
WAN UDP SIP_IPv4 5060 (SIP) 123.45.6.789 * 192.168.20.21 5062 SIP IPv4 redirect
WAN UDP SIP_IPv4 5060 (SIP) 123.45.6.789 * 192.168.20.22 5064 SIP IPv4 redirect
I think these inbound NAT rules should read:
Source Destination NAT
Interface Proto Address Ports Address Ports IP Ports Description
WAN UDP SIP_IPv4 * 123.45.6.789 5062 (SIP) 192.168.20.21 5062 SIP IPv4 redirect
WAN UDP SIP_IPv4 * 123.45.6.789 5064 (SIP) 192.168.20.22 5064 SIP IPv4 redirect
You NAT destination ports and you do not know the source port (could be random). Also, 123.45.6.789 should be "this firewall" or "WAN address".
Thank you for your patience meyergru,
It is encouraging to hear your VoIP works with the latest opnsense changes, there must be hope for mine too. :)
Following your initial suggestion I had removed port translation for SIP, like this: Source Destination NAT
Interface Proto Address Ports Address Ports IP Ports Description
WAN UDP SIP_IPv4 5060 (SIP) 123.45.6.789 5062 192.168.20.21 5062 SIP IPv4 redirect
WAN UDP SIP_IPv4 5060 (SIP) 123.45.6.789 5064 192.168.20.22 5064 SIP IPv4 redirect
Just now, I additionally removed the 5060 SIP port for the ISP's source address: Source Destination NAT
Interface Proto Address Ports Address Ports IP Ports Description
WAN UDP SIP_IPv4 * 123.45.6.789 5062 192.168.20.21 5062 SIP IPv4 redirect
WAN UDP SIP_IPv4 * 123.45.6.789 5064 192.168.20.22 5064 SIP IPv4 redirect
123.45.6.789 is meant to show the obfuscated public address of my WAN interface. SIP ports 5062 and 5064 are hardcoded respectively in the two VoIP phones, so these are the ports they use in their SIP communication with the ISP registrar.
Despite the above changes I am not able to place calls, although I continue to be able to register and receive calls.
Quote from: fbantgat7 on September 02, 2024, 01:10:38 PM
Thanks meyergru,
I added two Outbound rules one for each phone. I now have:nat on pppoe0 inet proto udp from 192.168.20.21 to <SIP_IPv4> -> (pppoe0:0) static-port
nat on pppoe0 inet proto udp from 192.168.20.22 to <SIP_IPv4> -> (pppoe0:0) static-port
I saw this now only:
Why those overly specific rules? You just have to have outbound NAT for your VoIP phones with static ports. That is, those NAT rules must beplace before any other outbound NAT rules (hence why "Hybrid Outbound NAT generation", but I showed you rules that do not have any specific ports or destination IPs in them. Mind you: The outbound NAT is needed for RTP as well and that does NOT go to SIP_IPv4.
If you have access to captures you also might want to check if your INVITES don't exceed 1500 bytes. Fix when it does by disabling CODECS that are not used and diversion headers and stuff like that. Also check the INVITES for IP addresses other than what the phone itself has. Disable NAT Traversal stuff and STUN on the phones might help since they habitually change private IP addresses to public ones. You want your INVITES to contain the real IP addresses of the phones since the SBC of the operator will fix that. If the contents are confusing to the SBC of the operator, it might reply to wrong IP Addresses or ports. Resulting in you not getting replies to INVITES. That's why the majority of operators advice to disable SIP-Helpers Like SIP-ALG and others. OPNSense doesn't have SIP-Helpers by default so you should be good there.
If configured right the SIP Proxy (SIP-ALG Plugin os-siproxd) could also help you by the way. But I never tried anything with it. So I would use it as last resort.
Normally, it shouldn't be necessary to create manual NAT rules for SIP Phones. They should register with SIP, which also creates a dynamic NAT translation and Firewall Pinhole (if not restricted). In an INVITE, there would also be signalling information about what ports and IP's to be used for RTP (the actual Audio) and those should also be the real IP addresses. The best suggestion I have is to leave that process to be handled automatically.
I as a VoIP engineer find OPNSense and PFsense a most excellent Edge Firewalls straight out the box.
Quote from: BoodahsFever on September 03, 2024, 11:50:03 AM
Normally, it shouldn't be necessary to create manual NAT rules for SIP Phones. They should register with SIP, which also creates a dynamic NAT translation and Firewall Pinhole (if not restricted). In an INVITE, there would also be signalling information about what ports and IP's to be used for RTP (the actual Audio) and those should also be the real IP addresses. The best suggestion I have is to leave that process to be handled automatically.
I as a VoIP engineer find OPNSense and PFsense a most excellent Edge Firewalls straight out the box.
That is incorrect: The SIP call itself passes usual NAT rules, sure. But for example, contained in it, the RTP ports that should be used are advertised. If the firewall decides to translate one port to another (as is the default with NAT), it cannot be reflected in the SIP packet (unless you make use of a SIP proxy).
Almost any guide on how to enable a SIP phone behind OpnSense willl tell you to use "static port" on outgoing NAT and at least for me, this was neccessary.
Have here several Gigasets, no special gym needed to make them work. Just some FW rules to allow for the IPs of the provider and done here...
Quote from: chemlud on September 03, 2024, 02:21:41 PM
Have here several Gigasets, no special gym needed to make them work. Just some FW rules to allow for the IPs of the provider and done here...
Exactly and this is how it is supposed to work. Normally you should not have the need for static translations. I have a few OPNSense boxes installed at large sites without the need to do anything special to make phones work.
I hope I explain this right.
Outbound calls:In a line like this in the Message body of an INVITE announces on what port it would like to receive RTP on (the firewall doesn't do anything with this):
m=audio 52154 RTP/AVP 8 0 9 101Likewise in the subsequential 200 OK message the other party announces what port it wants to receive RTP on:
m=audio 58130 RTP/AVP 8 101Also this is part of Codec Negotiation.
After the 200 OK and some other messages the phone will start sending RTP to the port announced in the 200 Ok from the port itself announced in the First INVITE. Opnsense should create that PAT/NAT Translation and Firewall PINHOLE where the other party can return the RTP to because that stream is being sent and allowed. No proxy required on the Edge side of things.
RTP packet Source port 52154 Destination port 58130
Inbound Calls:For INBOUND calls the server Sends the INVITE to the Address and (randomised by Edge Firewall/Router) Port that are in the SIP Register cache table maintained by the SBC of the operator. The principle is the same but this time the SIP Server announces it's port in the INVITE and the Client in the 200 OK. The client will again start the RTP stream and so on.
The proxy is in this case on the operators site. And I call this an SBC (Session Border Controller).
The only time I can imagine you would need a static NAT translation is when you don't use the SIP Register mechanism and the SIP Server use static registrations.
Just search the forums for "VOIP", "static port" and see how many people have had problems that have been fixed by that (me being one of those and I know others who use Fritzboxen).
I also know how SIP is supposed to handle things and how a firewall "should" do it. Matter-of-fact, I run my own OpenSIPS server. But SIP is a very old standard and things have evolved quite a bit since I started using it in the early 90s.
I think, you have three choices to make things work even on corner cases:
1. Do not allow arbitrary port translations (aka enable "static port") on outbound NAT
2. Employ a SIP proxy that monitors and rewrites SIP messages according to the translations in use
3. Use IPv6
Other than that, maybe you are lucky that your equipment and/or VoIP provider works without using one of the "guaranteed" ways from above.
At least I am quite sure that using "static port" will do no harm whatsoever. If you have an explanation or a solution for the OP on why his outgoing SIP calls do not work, go ahead.
Up to this point, I still think that the outbound NAT rules are wrong one way or another. Maybe the DNS entries do not map to the suspected IPs, maybe it is even worse and he is behind CGNAT or other type of double NAT.
Quote from: meyergru on September 03, 2024, 03:56:46 PM
At least I am quite sure that using "static port" will do no harm whatsoever. If you have an explanation or a solution for the OP on why his outgoing SIP calls do not work, go ahead.
Up to this point, I still think that the outbound NAT rules are wrong one way or another. Maybe the DNS entries do not map to the suspected IPs, maybe it is even worse and he is behind CGNAT or other type of double NAT.
I agree it won't harm. I just say normally in a B2BUA with symmetric signalling situation which is pretty common these days it shouldn't be needed. Removing the NAT config could also remove errors in his config.
I agree it is probably some misconfiguration and that could also be a misconfiguration of the phone. If he is behind cgnat or double Nat I wish him the best of luck. I hope he has IPv6 working in that case.
Thank you guys for all your help! :)
TL;DR: In my case the problem has been with the way the recent kernel changes have affected how the MTU is being processed. Just to confirm, in my case I did not need a "static port", or additional outbound NAT beyond the automatically created NAT rules when setting Port Forwarding. This is because in this case the phones and ISP's SIP gateway are 'smart' enough to acknowledge the SIP/RTP ports requested by the phones and use them reliably.
Since I started looking at this problem, I've spent some time checking firewall rules and trying all the different settings meyergru suggested, including "static port" for outbound NAT. I tried setting a Hybrid Outbound rule with:
Destination address: any
and when it made no difference with:
Destination port: the ISP's SIP/RTP addresses.
Still no success. I also removed port translation as recommended and took the opportunity to tidy up addresses in my aliases for the ISP's SIP/RTP gateway. Without a usable VoIP setup I was heading back to the 24.7 firmware, but as a last thing before calling it a day I decided late yesterday to use IPv6, thinking this would allow me to do away with NATing and port forwarding complications, or potential configuration errors. At this point I discovered with IPv6 the phone registration would fail altogether! So instead of the IPv4 issue of just outgoing calls failing, with IPV6 the whole lot stopped working. I suspected an opnsense MTU regression problem (has happened before), but ran out of time.
This morning I checked the WAN MTU and sure enough things looked different ...igb0: flags=1008843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST,LOWER_UP> metric 0 mtu 1508
options=4802028<VLAN_MTU,JUMBO_MTU,WOL_MAGIC,HWSTATS,MEXTPG>
[snip ...]
pppoe0: flags=10088d1<UP,POINTOPOINT,RUNNING,NOARP,SIMPLEX,MULTICAST,LOWER_UP> metric 0 mtu 1508
description: WAN (wan)
Then I saw BoodahsFever's post (thank you!), mentioning the INVITE packet size and how it can get larger with ever more RTP codecs added to it. So I changed the phone settings to advertise only two codecs suitable for my ISP, which reduced a 1512 byte WAN packet down to 1376 and from then on outgoing calls became possible again. :D
NOTE: Some months/years ago I had to keep setting the MTU manually using ifconfig on igb0 to 1508, so that the WAN PPPoe0 MTU would have a full IPv4 ethernet payload of 1500. Now ifconfig displays the ethernet MTU as 1508, but ping tests indicate it is still 1500.
Either way, should packet fragmentation break VoIP?
Something in the 24.7.x changes has caused my specific problem, but more applications/devices may be lurking out there similarly affected. Is this something to be addressed in a future update?
Quote from: meyergru on September 03, 2024, 03:56:46 PM
Up to this point, I still think that the outbound NAT rules are wrong one way or another. Maybe the DNS entries do not map to the suspected IPs, maybe it is even worse and he is behind CGNAT or other type of double NAT.
Perhaps my reasoning was wrong, but I was thinking all that changed to cause the phones to stop calling out was the opensense firmware update. It could be the updates allowed for hidden errors in my configuration to show up, and/or perhaps something else was clashing with the latest firmware changes. As it happened it was an MTU issue causing fragmentation.
Either way, thank you meyergru for your config suggestions, it helped me improve and tidy up my rules.
Interesting. Even in two ways:
1. I never had imagined that a SIP INVITE or REGISTER could be that big... so BoodahsFever was right!
2. I use MTU 1508 over PPPoE with a VLAN as well in order to have net 1500 MTU and my ifconfig on my pppoe0 shows 1508. The underlying physical interface is igc0 (on which there is the VLAN) has even 1512.
I have configured this all manually before 24.7 and checked via a PMTU tool from here: https://www.baeldung.com/linux/maximum-transmission-unit-mtu-ip that my actual MTU really is 1500 bytes. I have to say that when I tried this first, I noticed that the presumed "automatic" MTU inheritance never worked for me over pppoe0 -> vlanX -> igc0.
I re-checked again just now and the settings are still intact. If you rely on the automagic, this could have changed in the meantime. You probably would have noticed problems for some websites that have no PMTU discovery as well.
Quote from: fbantgat7 on September 03, 2024, 05:56:25 PM
Either way, should packet fragmentation break VoIP?
That depends on the Operator. Some allow for packet fragmentation on Access Realms others don't. Most Operators handle larger then 1500 bytes packets with the advice to use TCP instead of UDP. Offcourse they should then allow for TCP.
Oracle (Acme Packet) SBC's now ship with the advise to use TCP for SIP period as INVITES grow bigger and bigger these days.
Quote
I never had imagined that a SIP INVITE or REGISTER could be that big.
SIP REGISTER Messages usually don't get very big. That's why fbantgat7 phones where still registering causing me to suspect a MTU issue.
QuoteYou probably would have noticed problems for some websites that have no PMTU discovery as well.
I think this affects UDP only. But I can't be sure.
@fbantgat7
Is your OPNSense install virtualized and using a virtual NIC connected to an ONT?
No, opnsense is installed on bare metal and its igb0 NIC is connected directly with a CAT6 cable to the ONT.
I've had an on & off MTU configuration issue with opnsense for years now. It used to be the case when I'd set WAN's PPPoE MTU on the GUI to 1508, this would push the MTU of igb0 to 1508 to allow 8 bytes for the PPPoE header+ID tag and leave 1500 bytes for the payload through WAN. Unlike meyergru I don't use VLAN, but if I did the VLAN tag would add another 4 bytes for the VLAN tag, increasing the frame size to 1500+8+4=1512.
At some point the WAN interface settings GUI no longer worked this way, so I'd set the MTU for igb0 manually to 1508 on the console, which would provide a 1500 MTU for the WAN interface. Annoyingly, I'd have to repeat this operation after each update/upgrade of the opnsense firmware - the setting would not stick over a reboot.
For the last few months, up and including 24.7, the MTU setting had started to stick, so I didn't have to look into (re)setting it manually.
Anyway, after I saw the phone would also fail to work over IPv6 and spotted errors on the phone logs mentioning something about no PDU received, I thought it must be an MTU issue again. I checked with ifconfig and the Interfaces GUI details and was surprised to see pppoe0 showing an MTU of 1508, instead of 1500. The igb0 link also showed MTU 1508, as it should. Both couldn't be right. However, ping tests on the router showed the WAN MTU was indeed 1500:
~ # ping -6 -c 1 -D -s 1452 google.com
PING(1500=40+8+1452 bytes) 2001:XXXXXXXXXXXXXX --> 2a00:1450:4009:827::200e
1460 bytes from 2a00:1450:4009:827::200e, icmp_seq=0 hlim=120 time=5.871 ms
~ # ping -6 -c 1 -D -s 1453 google.com
PING(1501=40+8+1453 bytes) 2001:XXXXXXXXXXXXXX --> 2a00:1450:4009:827::200e
ping: sendmsg: Message too long
ping: wrote google.com 1461 chars, ret=-1
The above ping tests are no different when run with firmware 24.7, although with 24.7 the pppoe0 MTU is displayed correctly as 1500 not 1508.
Here is what I fail to understand:
Despite the displayed MTU size anomaly, ping tests indicate an effective WAN MTU of 1500 bytes with both 24.7 and 24.7.x - so why does a larger INVITE datagram go through fine on 24.7, but fails to do so with 24.7.x? What else is at play here?
Yeah something seems off with Either MTU or Packet Fragmentation. But it is hard to pinpoint.
@home I run OPNSense as VM inside Proxmox. I know not recommended and stuff. But I find it convenient.
Before I was connected via a Capble ISP and the cable modem in bridged mode. No PPPoE just Ethernet with MTU of 1500 and DHCP/SLAAC for IPv4/6. The Wan Facing interface was also virtual (VirtIO driver). No problems ever with Packet Fragments. I work for a Telco so I can monitor our SBC's and we accept fragmented packets. So packets larger than 1500 bytes are known to arrive and be processed by our SBC.
After that GlassFiber was connect at my place so I changed ISP to make use of that. Now it is PPPoE and my setting are the same as meyergru. When rebooting OPNSense or Proxmox I used to get your issues as well. Then I would reboot the ONT and everything started working again. I could see the packets leaving OPNSense but they never arrived at the SBC. Invisible to see where the packets where dropped. I thought maybe a Auto Neg issue between Proxmox and the ONT. Note this was also UDP packets larger than 1500 bytes.
After a while I was fed up with that. So I performed a PCI passthrough on proxmox letting OPNSense manage the Wan interface and the problem is gone. Everything works as expected again. It seems weird to me as the Operating System is responsible for MTU and Packet Fragmentation (L3) as the OS controls the IP stack. The network driver should be responsible for L2.
I am having a hard time wrapping my brain around this.
I'm on release 24.7.4_1 now, but the same RTP fragmentation problem remains. :(
I noticed something with ping which I can't recall if it was the same on 24.7. When I ping using IPv4 to check MTU size, it complains about the packet size (unless I use sudo):
~ $ ping -4 -c 1 -D -s 1472 www.google.com
ping: packet size too large: 1472 > 56: Operation not permitted
However, no such problem with IPv6:
~ $ ping -6 -c 1 -D -s 1452 www.google.com
PING(1500=40+8+1452 bytes) 2001:XXXXXXXXXXX --> 2a00:1450:4009:81f::2004
1460 bytes from 2a00:1450:4009:81f::2004, icmp_seq=0 hlim=120 time=6.121 ms
--- www.google.com ping statistics ---
1 packets transmitted, 1 packets received, 0.0% packet loss
round-trip min/avg/max/stddev = 6.121/6.121/6.121/0.000 ms
Could someone please confirm if ping packet size is meant to have this constraint on IPv4 only, or if it is a result of the later releases?