Invalid argument and no SYN after update to 25.7

Started by Mirio, July 27, 2025, 04:22:53 PM

Previous topic - Next topic
Hi all,
Yesterday, I performed an upgrade from OPNsense 25.1 to 25.7 via the web UI. I immediately encountered issues regarding the addition of new nodes.

What is the issue?
The issue is that when I try to add a new node (new MAC address and IP), the DHCP packet arrives successfully. However, any other packet (TCP/ICMP/UDP, etc.) seems to be correctly sent and received by OPNsense (verified via packet capture), but the opnsense node does not respond. Meanwhile, all older nodes (configured before the migration) continue to function without issues. Firewall rules are exactly the same for both old and new nodes.

A particular curiosity I noticed is that new nodes are not pingable from OPNsense and return the error **"invalid argument"**.

Here's a small snippet for reference
# opnsense > ping -c 2 192.168.50.189
PING 192.168.50.189 (192.168.50.189): 56 data bytes
ping: sendto: Invalid argument
ping: sendto: Invalid argument

--- 192.168.50.189 ping statistics ---
2 packets transmitted, 0 packets received, 100.0% packet loss

# opnsense > ping -c 2 192.168.50.100
PING 192.168.50.100 (192.168.50.100): 56 data bytes
64 bytes from 192.168.50.100: icmp_seq=0 ttl=64 time=0.141 ms
64 bytes from 192.168.50.100: icmp_seq=1 ttl=64 time=0.149 ms

--- 192.168.50.100 ping statistics ---
2 packets transmitted, 2 packets received, 0.0% packet loss
round-trip min/avg/max/stddev = 0.141/0.145/0.149/0.004 ms

What types of tests and actions have I already done?
- I've manually migrated from ISC to DNSMASQ (DHCP+DNS)
- Flush ARP table
- Full restart of OPNSENSE and 'target' VM
- Firewall rules reviewed (floating and automatic included)
- Search on the forum and google

Any idea?
Thank you

July 27, 2025, 04:41:30 PM #1 Last Edit: July 27, 2025, 04:43:03 PM by meyergru
The 'ping: sendto: Invalid argument' indicates that the packet cannot be sent, e.g. because the NIC is defective or has no link or the route does not exist (e.g. wrong netmask). I assume 192.168.50.100 is OpnSense itself?
Intel N100, 4* I226-V, 2* 82559, 16 GByte, 500 GByte NVME, ZTE F6005

1100 down / 800 up, Bufferbloat A+

No, the node 192.168.50.100 is a old node pre migration. Same network as .189.
How can I search for problems on the nic? Do i have to launch some cmd?

July 27, 2025, 05:54:23 PM #3 Last Edit: July 27, 2025, 05:57:29 PM by meyergru
It would be really strange if the NIC was the culprit when you can ping some clients but not others on the same network.

I would think that this is a routing issue. Please show the output of "netstat -nr -4" and anonymize the default line.
Intel N100, 4* I226-V, 2* 82559, 16 GByte, 500 GByte NVME, ZTE F6005

1100 down / 800 up, Bufferbloat A+

Yeah, below the the routes

Routing tables

Internet:
Destination        Gateway            Flags        Netif Expire
default            xxx.xxx.xxx.xxx    UGS            igc0
127.0.0.1          link#5            UH              lo0
....
192.168.50.0/24    link#3            U            vtnet0
192.168.50.1      link#5            UHS            lo0
....


# ifconfig vtnet0

vtnet0: flags=1088843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST,STATICARP,LOWER_UP> metric 0 mtu 1500
    description: LAN50 (opt1)
    options=80028<VLAN_MTU,JUMBO_MTU,LINKSTATE>
    ether [REDACTED]
    inet 192.168.50.1 netmask 0xffffff00 broadcast 192.168.50.255
    media: Ethernet autoselect (10Gbase-T <full-duplex>)
    status: active
    nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>

You cannot view this attachment.



I'm not sure if it's relevant, but OPNsense is hosted inside Proxmox (yes, proxmox needs a IP in the network for a specific reason), and this interface is configured as follows on the Proxmox side:
auto vmbr50
iface vmbr50 inet static
    address 192.168.50.210/24
    bridge-ports none
    bridge-stp off
    bridge-fd 0

July 27, 2025, 07:08:39 PM #5 Last Edit: July 29, 2025, 12:20:10 AM by meyergru
O.K. - that Proxmox detail is relevant. At least, if you did not follow all advice here, especially the parts about network "hardware".

Also, how would vmbr50 work without any bridge ports? It is connected to nothing but the Proxmox host and its guests.
Intel N100, 4* I226-V, 2* 82559, 16 GByte, 500 GByte NVME, ZTE F6005

1100 down / 800 up, Bufferbloat A+

The concept behind it is quite straightforward, although I'm not sure if it's the right choice. If that's the case, I'd appreciate it if you could let me know if there are more tailored or specialized options for OPNsense. All the VMs in that network have only one NIC within that bridge, and OPNsense manages all the traffic acting as a gateway.


July 27, 2025, 09:17:53 PM #7 Last Edit: July 29, 2025, 12:20:44 AM by meyergru
If vmbr0 is a bridge that handles VM clients only, like depicted in your drawing, it is O.K. to use no physical NICs. But in that case, no physical clients will be able to communicate via IPs in subnet 192.168.50.0/24 (except Proxmox host and guests), since this is a pure Proxmox-internal network segment.

Depending on the age of your previous OpnSense installation, it may well be that a missing tuneable mentioned in the link I gave above may be to blame.

However, I would try to rule out other causes by rolling back to a previous OpnSense release to verify if it really is the 25.7 version that causes these problems. Being a VM, this seems feasible.
Intel N100, 4* I226-V, 2* 82559, 16 GByte, 500 GByte NVME, ZTE F6005

1100 down / 800 up, Bufferbloat A+

After some deep dive into it, i've resolved it downgrading the kernel for proxmox to 6.8.12-7-pve.
Thank you!

Did you use the optional 6.14 kernel series or was it just the newest 6.8.12-13-pve that broke it?

Did you report that problem to Proxmox?
Intel N100, 4* I226-V, 2* 82559, 16 GByte, 500 GByte NVME, ZTE F6005

1100 down / 800 up, Bufferbloat A+

It was the 6.8.12-13-pve. Yes i've found the issue on the pve forum