Routing loop between LANs via L3 switch

[Azmodes]

I recently got a layer-3 switch to play around with and I ran into a confounding issue while setting up inter-VLAN routing on it.


The relevant network topology is this:

Main router (OPNsense 24.1.10_3-amd64; LAN interface 192.168.10.1/24)

connected to

Main (layer-2 smart) switch (connected to many other hosts on the 192.168.10.0/24 network)

connected via 2-port LAG to

Layer-3 switch (a Cisco SG300-28P; SVI 192.168.10.113/24, the uplink LAG has been set up as a trunk/tagged for all VLANs known to the L3 switch)

On the L3 switch I then configured VLAN50 with SVI 192.168.50.1/24 and enabled DHCP for that subnet that hands out IPs as well as 192.168.50.1 as the default gateway.

Static routes
On main router (manually created):

Code Select Expand

192.168.50.0/24 192.168.10.113/24
(created a gateway entry for 192.168.10.113/24)

On L3 switch (obv. all automatically created):

Code Select Expand

0.0.0.0 192.168.10.1         VLAN1
192.168.10.0 Directly Connected VLAN1
192.168.50.0 Directly Connected VLAN50

I configured a single port on the L3 switch as access/untagged for VLAN50 and connected a laptop to it in order to test things. The laptop receives an IP/gateway just fine and is even able to ping hosts on the 192.168.10.0/24 net. However, when I try to ping the router at 192.168.10.1, I run into a loop. Traceroute tells me the traffic is being bounced back and forth between 192.168.50.1 and 192.168.10.1 and never terminating. Is this some sort of asymmetric routing issue? The ping works fine from 192.168.10.1 to the VLAN50 host.

Furthermore, not counting the loop, only ICMP traffic seems to be working to the 192.168.10.0 net (although the culprit here appears to be an issue with my OPNSense firewall rules on the LAN interface; Default deny / state violation rule is being triggered for traffic that should be explicitly allowed).

EDIT: One more caveat: I can access the OPNsense webGUI at 192.168.10.1 from the VLAN50 host, but there is no internet connectivity as all WAN IP routing is stuck in the aforementioned loop and/or because of the overzealous firewall.

[Azmodes]

Okay, I've at least figured out why there was no internet connectivity for VLAN50. Had to add an outbound NAT rule for that subnet. Which makes sense in hindsight, since the automatically created NAT rules only apply to the subnets directly connected to the OPNsense machine.

The routing loop kind of... vanished after I played around with the firewall rules. Can't really say why, but good enough.

What still does not work is accessing anything in the 192.168.10.0 net from a VLAN50 host (apart from the GUI at 192.168.10.1). Looking at the FW logs, it tells me that TCP traffic coming from 192.168.10.0 hosts to VLAN50 is being blocked by the generic Default deny / state violation rule, even though I have specific rules on that interface to allow 10<->50 traffic of any kind. ICMP works both ways, though, as does -evidently- UDP, since I can send DNS requests to my Piholes on the 10 net.

[Azmodes]

What still does not work is accessing anything in the 192.168.10.0 net from a VLAN50 host (apart from the GUI at 192.168.10.1). Looking at the FW logs, it tells me that TCP traffic coming from 192.168.10.0 hosts to VLAN50 is being blocked by the generic Default deny / state violation rule, even though I have specific rules on that interface to allow 10<->50 traffic of any kind. ICMP works both ways, though, as does -evidently- UDP, since I can send DNS requests to my Piholes on the 10 net.

I finally got this to work by checking Static route filtering in Firewall > Settings > Advanced.