Configuration as "Switch" on VM

[jurjevic]

Hi,

I'm running opnsense on VM (Unraid/QEMU) and want to use the network interfaces as a switch.

While there is a good documentation for a non vm version, it seems not to work running inside a vm (I double-checked the docs).

Let me give some details.

---
VM
---

Physical Interfaces:
4x RealtekCo. RTL8125 2.5GbE Controller

Virtual Interfaces:
2x virtio-net / br0

-----------
opnsense
-----------

Interfaces:
- LAN (lan) ---> bridge0 (Bridge)
- Mainboard (opt2) ---> vtnet0
- OPT1 (opt1) ---> re3
- OPT2 (opt3) ---> re2   
- OPT3 (opt4) ---> re2
- OPT4 (opt5) ---> re1
- WAN (wan) --> vtnet1

WAN (wan) has a IP from DHCP server which runs on my router. So WAN is just bridged with my real LAN. I use the WAN to access the opnsense.

LAN (lan) is the opnsense bridge0 with assigned Mainboard, OPT1, OPT2, OPT3, OPT4 device an local link is off.

------------------------------------------
So the goal is, to build a switch 4 physical and 1 virtual nic. I want to attach my notebook to one phyical interfaces and it should be conncted to my original LAN automatically.

Unfortunatelly it doesn't work. I have no clue and need some help.
With "systat -ifstat" I just see "in" traffic for bridge0, but no "out" traffic.

Does somebody has a idea what I'm doing wrong?

Thx for any help.

[sphbecker]

OPNsense is honestly not designed to be a switch, so you may not get great performance even on physical, virtual will be even more of a challenge. I can also tell you from my own experience that running an OPNsense VM on unRaid, using vNICs had unusable performance. My understanding is that was some kind of conflict with the version of FreeBSD currently used by OPNsesne running on KVM, so it's possible that issue has been fixed.

I ended up using SR-VOI to physically assign the network ports to the VM. If that is possible for your configuration, I highly encourage you to do that. If not, see if you can completely assign the PCIe/USB device to the VM. Even if the above vNIC issue has been fixed, hardware assignment will result in the best possible performance.

Now to the OPNsense question. To make OPNsense a switch, don't assign anything to the WAN or LAN interfaces, just leave them unassigned. That simplifies issues with default services running on those interfaces. Assign all your hardware NICs (or vNICs) to OPT interfaces, then create a bridge interface in OPNsense that uses all those OPT interfaces.

Having said all of this, I don't think you realize how much CPU power would be needed for what you are trying to do. Whatever your need is, it would likely be much better served with an <$80 gigabit switch. If your goal is to use faster than gigabit interfaces, then it is unlikely you will ever get that level of performance on an OPNsense bridge without burning crazy amounts of CPU cycles.

[sphbecker]

The more I think about this, if you really want to turn an unRaid server into a switch, you should look at defining a bridge interface on the unRaid OS itself (not supported with the GUI but should be possible with a Linux script running at a startup). This is the kind of thing that will work best running directly on real hardware, not a VM.

[jurjevic]

Okay... I will do it without opnsense and directly on Unraid.

Thx sphbecker