Bridge Looping Issue: Need Help with STP Activation

[skydiablo]

Code: [Select]

┌───────────────────────────────────┐
│ Firewall 1                        │
│            ┌───────────┐          │
│     ┌──────┤ vlan0.0.50├──┐  ┌────┤
│     │      └───────────┘  └──┤eth0├────────────────────────┐
│ ┌───┴───┐                    └────┤                        │
│ │       │                         │                        │
│ │bridge0│                         │                        │
│ │       │                         │                        │
│ └───┬───┘                    ┌────┤                        │
│     │      ┌───────────┐  ┌──┤eth1├──────────┐             │
│     └──────┤ vlan0.1.50├──┘  └────┤          │             │
│            └───────────┘          │          │             │
│                                   │     ┌────┴─────┐  ┌────┴─────┐
└───────────────────────────────────┘     │          │  │          │
                                          │  switch1 │  │  switch0 │
┌───────────────────────────────────┐     │          │  │          │
│ Firewall 2                        │     └────┬─────┘  └────┬─────┘
│            ┌───────────┐          │          │             │
│     ┌──────┤ vlan0.0.50├──┐  ┌────┤          │             │
│     │      └───────────┘  └──┤eth0├──────────┼─────────────┘
│ ┌───┴───┐                    └────┤          │
│ │       │                         │          │
│ │bridge0│                         │          │
│ │       │                         │          │
│ └───┬───┘                    ┌────┤          │
│     │      ┌───────────┐  ┌──┤eth1├──────────┘
│     └──────┤ vlan0.1.50├──┘  └────┤
│            └───────────┘          │
│                                   │
└───────────────────────────────────┘

Hello, I have a problem with a loop that I get with an HA configuration over two OPNsense firewalls. I have a bridge on each firewall, and each bridge is bridged to a VLAN via two paths. As can be seen from the diagram, a loop must occur here, unfortunately I need this setup exactly like this. As soon as I set up the setup like this, I have a loop and everything comes to a standstill. Even various attempts to activate STP have not clue, which I also don't really understand.
In terms of HA, I have built myself a sys hook: /usr/local/etc/rc.syshook.d/carp/10-handle-briges.
This script does an ifconfig bridge0 up/down according to the CARP status (MASTER or not MASTER). Basically, this works quite well, but unfortunately not when I restart the firewall. When booting up, CARP takes effect and the bridge is shut down, unfortunately in the following boot process it is detected that the bridge has dependencies, it is configured and thereby brought back up, and I am back to my loop.

Ideally, I should be able to counteract all of this with STP, but it's not working. Does anyone have any idea why not? The attempt to bypass the whole thing using a hook and bringing the bridge up and down is just a workaround; here, I would still need an idea of how to immediately bring the bridge back down in case it is brought back up, depending on the CARP status. My current approach is to write a file (/tmp/carp-status.flag) to check in another script whether the bridge should be up or down, and then to correct this immediately if necessary. Unfortunately, I haven't found a hook yet that I could use as soon as the bridge is brought back up. Maybe with the devd stuff? But I haven't figured out yet how to register for an event here, maybe somewhere in /usr/local/etc/inc/plugins.inc.d/?

Perhaps someone can give me an idea or question everything again and guide me in the right direction?

Greetings, Volker

[Patrick M. Hausen]

Fundamentally STP works just fine in FreeBSD/OPNsense. I suspect the problem lies with the VLANs.

If you put the bridge across two VLAN interfaces - do your switches perform PVST? (Per VLAN Spanning Tree?)
And I am not quite sure this is even standardized enough to work seamlessly across vendors. I might be wrong, though. But the last time I dealt with this we were still "Cisco only".

So if VLAN 50 is the only one you need in that topology, I'd try to change the switches from trunks to access ports with fixed VLAN 50 assignments and change the OPNsense ports to untagged, too. Then check again if STP works. It should.

HTH,
Patrick

[skydiablo]

Oh, sorry! It's the usual "rubber duck" thing. Thank you for talking about it. After I double-checked all the settings and realized that I hadn't correctly specified the STP interfaces in full, I couldn't detect any loop problems after the correction. Now it's working! Many thanks!

[skydiablo]

hmmm, i'm wrong, the issue is still there. STP isnt working and setting up the bridges up and down seems to be more complicated as first seen i have to write a more complex script to reattach all members back to bridge. sad

[Patrick M. Hausen]

Did you try STP with untagged interfaces?

[skydiablo]

for now, i have active "private VLAN" on switch0, it is working just now, but not really what i want. The layer 2 traffic is still being forwarded to switch0 as a result, which I would like to avoid.

what do you mean with "untagged interface" ? i have create an VLAN Interface (vlan0.0.50 - parent eth0), this will bring the vlan already as untagged packages into the bride, or i'm wrong?

[Patrick M. Hausen]

My theory is that STP does not work over VLANs with your combination of switch and OPNsense. I would try to use access ports instead of trunk ports if only a single VLAN is required. Why tag in OPNsense at all?

[skydiablo]

this shown schema is just a small part of the big picture, so i need to handle this with VLANs. i will refactor this problem with access ports only and will give you a feedback. but in the end, i should resolve it with VLAN instead.

mybe you have another tool for me to debug STP that will show why it will not work? some missing packages or what ever?

[Patrick M. Hausen]

I suspect missing PVST support on the FreeBSD side or incompatible interpretation of PVST ...