OPNsense 26.7.4 - VLAN traffic failing over X550/LAGG/bridge after upgrade

[merrins63]

I have got a question that really bugs me :

Why would you do this =>

I also have an Intel i226 2.5 GbE trunk, with corresponding VLAN interfaces bridged to the X550/LAGG VLAN interfaces.

Aren't you effectively Bridging 2x 2,5 Gbps with 2 x 10 Gbps ?!

What is the purpose of such a setup ??

The issue I'm trying to solve with this design is mainly about making the best use of the different network speeds available in my home.

I have a traditional 1 GbE wired network, which is perfectly adequate for many of my wired IoT devices. However, my wireless access points and newer switches are capable of 2.5 GbE.

I need my wired and wireless IoT devices to remain on the same VLANs and subnets, while allowing the 2.5 GbE side of the network to make use of the faster interfaces rather than forcing everything through the existing 1 GbE switching infrastructure.

I'm also planning to upgrade my Internet connection to 2000/200 in the next few months, and I already have several multi-gigabit devices that will be able to take advantage of that bandwidth.

Previously, I used the more traditional approach of a single 24-port Gigabit Ethernet switch. However, with affordable 2.5 GbE switches now readily available, I wanted to take advantage of the Cat6A cabling already installed throughout my house.

The important point is that this design isn't intended for redundancy. The objective is to utilise the available 1 GbE and 2.5 GbE links efficiently while keeping devices within their existing VLANs.

Using bridged VLAN interfaces on OPNsense allows me to achieve that design.

Hopefully that explains the reasoning behind my setup a little better.

[RES217AIII]

Even if my post doesn't help you solve the specific problem, I wanted to write because I'm trying to understand your approach. You never stop learning, and I might get some ideas to rethink my own network design.

As I understand it, you are trying to get the most out of your existing 1G and 2.5G components. However, the current issue following the upgrade is a direct result of software bridging on the OPNsense.
In my opinion, using OPNsense purely as a bridge or Layer 2 switch fails to leverage its capabilities and doesn't align with its intended design.

Using the firewall as an L2 switch to connect these two worlds within the same VLANs creates a highly complex stack involving LACP, VLAN tagging, BSD bridges, and firewall rules.

Even if the problem is resolved, the solution remains fragile, places unnecessary load on the firewall CPU, and introduces extra latency.

To meet your bandwidth requirements, how about considering these solutions instead?

1.  A central 2.5G/10G switch handles all L2 switching in hardware. This offloads the OPNsense, allowing it to connect to the network via a clean trunk.
or
2.  Separating the 2.5G devices into their own subnets at Layer 3. This allows you to use OPNsense exactly as it was designed to be used.

Communication (mDNS, SSDP) between the networks can be ensured using a multicast relay, for example (https://forum.opnsense.org/index.php?topic=52705.0).

[Patrick M. Hausen]

The FreeBSD bridge has been rewritten from scratch and works perfectly fine. There is nothing wrong with this design if implemented correctly.

[RES217AIII]

To reiterate: My focus is on understanding the situation. There have already been several discussions on the forum regarding the use of OPNsense as a bridge.
And as I have learned, there are circumstances that make bridging necessary—though it remains only the second-best option.
Even though FreeBSD's bridge mode can work well, I do not understand why one would want to use it.
If you are building a network and planning network segmentation for valid reasons, the setup looks like this:

ISP – OPNsense Router – VLAN Trunk – Switch Segmentation – Clients in different network segments.

You could certainly make it more complex by separating the routing and firewall functions, but what advantage would that offer in the context of the described setup and requirements?

[Patrick M. Hausen]

Sometimes it's just how your apartment or office is built, the location of the "Internet socket", and so on. E.g. years ago in the office of my wife there was a single network connection to a small cabinet on which the laser printer was placed and it was decided that the new FreeNAS was to be put next to the printer. So I bridged the two ports of the NAS to connect NAS and printer to the network via that single line - why not? Instead of messing with an extra switch, needing yet another power outlet etc.

The OP might tell us their reasons :-)