Firewall (OPNsense) vs L3 Switching

[Ben.]

Hi,

I currently have an OPNsense instance running as my firewall and router for 3 VLANs.
My plan is to add 2 additional VLANs and replace the current Netgear Switches by a Ubiquity PoE L3 Switch.

So I have a basic question which you can maybe help me to answer:

If all my VLANs are fully separated from each other, I might only benefit from faster inter-VLAN traffic.
Even if I permit access from VLAN 1 to VLAN 2 (e.g. trusted to IoT), all traffic will go through OPNsense for "evaluation" as the L3 switch is not doing "firewalling".

Is that correct?

I want to better understand which device in my LAN needs to have more power. Is it worth investing in the L3 switch or should I rather replace the OPNsense device (which needs to handle a 100 MBit connection and is doing fine)?

Thanks for your thoughts.

[bimbar]

Either the switch does routing, or the firewall does.

It is not the case that the switch does route and the firewall firewalls those packets the switch routes.

[Ben.]

It is not the case that the switch does route and the firewall firewalls those packets the switch routes.

Yes, if the switch does the routing, the firewall won't see any of the traffic.

But if I try to separate each VLAN from each other, the switch can't do the routing, as it would pass all traffic between the VLANs, right? There are no rules to block ports in a L3 switch.

So I would have to disable L3 switching and run it as a L2 switch. This would then make the switch pass all traffic through the firewall, right?

[Patrick M. Hausen]

Right.

[Ben.]

Thanks for the confirmation.

[gpb]

Was curious about this so ran an iperf via switch and then passing through the router (LAN to VLAN) on relatively low end gear to quantify this.  I found 948 Mb/s via switch and 939 Mb/s through the router.  No rules on the switch (it is an "L3" but while I originally hoped to do what you're wanting, this switch doesn't have the flexibility I needed).  Cheers.

[Ben.]

Ok, cool. So what did YOU want to achieve and what did you end up with?

My plan at the moment is to replace the current OPNsense system by a stronger one and extend it by a SFP+ card so remove it as the bottleneck.

I feel this is the best solution for me maybe. Option b) would be LAGG via several ports, but I guess SFP+ could be faster overall. Not sure.

[gpb]

I moved from EdgeRouter a couple years ago during one of their firmware update fiascos.  Thought I would test OPNsense with an HP T620 Plus thin client and later added the T730 (now I have a backup).  I'm more hobbyist/homelabber so not needing raw bandwidth (200/20 service for now).  I focus more on lower power devices and value vs higher-end/esoteric.  Good luck! 

[Ben.]

I feel you. I run a APU1D4 at the moment and am quite happy with it, even it is reaching its limits now with kids growing up and using more bandwidth as well as adding more services to OPNsense.

I have to re-do my LAN soon so I want to upgrade here and there as there will be a lot more devices on my LAN soon.

But yes, a thin client to replace the APU is definitely the plan, but I need one with PCIe slot...

[gpb]

Used HP Thin Clients are hard to come by at the moment and the prices are what they were (I paid 129 for my T730).  I happened to watch this video yesterday, might be useful even though it's a couple years old and the prices have risen...but still might provide some insight.

https://www.youtube.com/watch?v=eeAzqpHl7NA