Communication timeout across VLAN (ACK not forwarded)

[abe_s.]

Hi together!

I have a fresh install of OpnSense 20.1.6 on a Supermicro X10SDV-2C-TP8F, 6x GbE, 2x SFP+. The SFP+ are aggregated to lagg0. The following interfaces are defined:

• lagg0_vlan100: 172.16.1.254/23 Intranet
• lagg0_vlan401: 172.16.7.14/29 Doorphone

OpnSense has more or less its initial configuration. For each interface there is an "allow any" rule (cloned from the default "LAN" rules). No IDS/IPS, no plugins, or other fancy things.

I have a doorphone available at 172.16.7.9 (VLAN 401) which offers a web interface at Port 80. I have at least one client in the Intranet (VLAN 100) from which the web interface is not reachable.

Here come the weird things:

• If the Client 172.16.0.15 (VLAN 100) is connected via Wireless (Killer AX1650) any connection to the doorphone (WEB, Ping, etc.) does not work (see packet capture). Other wireless clients in the same VLAN which I have tested can reach the Doorphone's web interface.
• If the same client from above is connected via wired ethernet, the doorphone's web interface is reachable
• Other clients can also reach the web interface, no matter if wired or wireless
• OpnSense replaces a hardware router. With the hardware router, 172.16.0.15 can reach the web interface

Using Packet Capture functionality, I traced it down to an SYN-ACK not being forwarded from 401 to 100, resulting in connection timeouts.

I attached the two packet captures of the above interfaces. I ran two tests, one failing test from 172.16.0.15 (wireless) and one successful test from a different wireless client (172.16.1.1, remember /23)

I recognized the Ethernet frame of the SYN-ACK in the failing case has destination adress ff:ff:ff:ff:ff:ff in VLAN 401, whereas in the successful case the destination is the MAC of the OpnSense router. I re-ran the same test with the hardware router, and despite the fact that the SYN-ACK is also using MAC broadcast, it worked with the hardware router.

So, I'm at the end of my knowledge. If someone has an idea, I appreciate any input.

Stefan

[Maurice]

I recognized the Ethernet frame of the SYN-ACK in the failing case has destination adress ff:ff:ff:ff:ff:ff in VLAN 401

Yeah, that shouldn't happen. Broadcasting TCP is just wrong. OPNsense most likely drops these packets. Your old router might be more forgiving.

So you'll have to investigate when / why your door phone broadcasts these packets instead of sending them to the OPNsense MAC address. Might be a bug, might be misconfiguration.

Cheers

Maurice