So assuming you do 99% networking with OPNSense - what Switch and AP?

[SecCon]

I like to ask around for opinions before I buy stuff. Sometimes it may be a bit moot, other times it could spark an interesting dialogue ( or flame war).

Have this notion that if you do use something like OPNSense the rest of the network equipment do not really need to do anything and can be really stupid. Just link. Sure, POE might be nice to have but that is more hw support to me.

My current hardware is a Mikrotik switch and an Asus AP. I would like to exchange those for something that is less complex in the interface (Mikrotik has the SwitchOS that I use, but I am gong to sell that anyhow) and more able to be managed via OPNSense.

What Switches and AP's can be managed via OPNSense, if at all?  I read this: https://docs.opnsense.org/manual/how-tos/interface_wireless_internal.html and it looks promising, but would you have to get in to the AP for management, or can you do it all via OPNSense?
 
I realize you may not want a completely unmanaged switch or AP, since FW updates on those may provide better security.

What did you get and how integrated is it with OPNSense? 

[bartjsmit]

Although I'm not in the least interested in flame wars, I would like to make a counter argument if you'll indulge me:

For optimal security, a firewall should make for a hole in your networks. Very little traffic should originate from it, or go to it. Access should be through a separate management network and it should spend most of its time regulating traffic that flows through it. Any interface you add is another attack vector.

My switch and AP management is a mixture of Unifi and Netgear, completely outside OPNsense and I intend to standardise on the former eventually, once supply lines go back to normal.

Bart...

[Vilhonator]

You can use whatever switches and APs you want.

For Switch I would recommend something like Cisco SG350-P series switch and any router with AP mode support or pure AP.

If Cisco isn't your cup of tea, then another great option for home use is Mikrotik switches and APs https://mikrotik.com/ (just like Opnsense, mikrotik uses Router and Switch OS which you can install on any computer, though I think you need a license for all features and they include it on their products)

[SecCon]

@bartjsmit

Well, the plan is to put the OPNSense in the middle of the traffic. WAN <-> OPNSense <-> LAN. So all traffic going to and from Internet will go through it, as you say. As far as I see that is as few interfaces as can be made.

So why should I not use its routing capabilities when doing that? It will be on a VM so performance wise it can get whatever it wants in regard to cpu, ram and storage. I was pondering giving it its own box, but since I have been experimenting a lot with VM's the last couple of years it seems to me it should be stable enough. Only need to sort out the virtual switching.

What is required by an AP to be able to manage it via OPNSense, is that even possible?

I want as few management systems as possible.

[Vilhonator]

You can use any AP or router with AP mode support but if you are using pure AP, there might not be management and you need to quite possibly setup captive portal (https://docs.opnsense.org/manual/captiveportal.html). I have Asus RT-N56UB1 running in AP mode, and I can manage it's wifi password through it's own WebGUI, but NAT, routing and Firewall things are managed by my Opnsense.

AP's Don't have Firewall, DHCP and NAT which means that you can only change Wireless network name and password, if it is pure AP, depending on brand and what type of AP it is.
If you use router running in AP mode, you might be able to have MAC restrictions and disable ports which aren't in use etc. but you won't specifically manage them really, other than some security things like prevent people from accessing the network with just right password.

Reason why you should use Switch and avoid routing as much as possible on Opnsense, is because more routes and more ports are being used, is because it easilly overwhelms firewall and either causes network disconnections or slows down the speed.
Your PC might be extreme high end gaming PC, but still wouldn't be able to handle all the traffic, since CPU speed and it's features are more important, than Core and thread count of the CPU. Especially if you are running opnsense on a virtual machine.

Network with 1-10 computers isn't that heavy with just simple firewall filtering, but add routes, VLAN, WIFI and IPS/IDS into it, and it is pretty darn heavy to run on Virtual machine, unless you're using VM Ware or eSXI server or something like it and not hosting virtual instance on PC you use to play games and surf the web etc. Virtualisation isn't exactly sollution when you don't have dedicated server or machine to host those.

Virtual Machines aren't really solution for home use unless you have BITCHIN PC (for example 128GB ECC RAM with 28 core Intel Xeon CPU and 100Gb QSFP network card) they only reduce costs when you have machine which is capable to host multiple instances (because that's what you have to do) Now 1 machine with Intel I7 and 32GB ram is able to host 1 instance without problems, but try playing games while hosting virtual machine, and you notice my point.

Also you might have some compatibility issues with virtualisation when it comes to Opnsense, it's supported yes, but hardware compatibility outside server hardware is bit more limited on that area.

[SecCon]

I have Asus RT-N56UB1 running in AP mode, and I can manage it's wifi password through it's own WebGUI, but NAT, routing and Firewall things are managed by my Opnsense.

AP's Don't have Firewall, DHCP and NAT which means that you can only change Wireless network name and password, if it is pure AP, depending on brand and what type of AP it is.

That might be what I would like to do. Asus RT-AC87U here. Has some rudimentary FW and protection features that would be disabled when demoting it to AP. It is currently running my Lan while i sort out questions and readings. I might get another AP down the road, but looking at WiFi BW I don't need it.

Reason why you should use Switch and avoid routing as much as possible on Opnsense, is because more routes and more ports are being used, is because it easilly overwhelms firewall and either causes network disconnections or slows down the speed.

Well, I am familiar with the notion to do routing/bridging/vlan close to the devices. Then again, I may only be doing three vlans, one for management, one for known devices and one for unknown devices and IoT. Not there yet. WiFi would be in one of those. I have yet to nail down the structure of my SOHO Lan.

Your PC might be extreme high end gaming PC, but still wouldn't be able to handle all the traffic, since CPU speed and it's features are more important, than Core and thread count of the CPU. Especially if you are running opnsense on a virtual machine.

It's a server. In a rack. I am not really so stupid as to run OPNSense virtualized on a gaming desktop machine. I am running OPNSense on my Dual-CPU 24Core Xeon 3.47GHz dedicated ESXi server with 144GB DDR3 and 7TB of disk. Virtualizing I have given it 4 cores and 16GB Ram so far. So I am no worried about performance or drivers or compatibility. Worst case I can move it to my 32Core DDR4 server, but that has a little less RAM and is not used for virtualization. My Lan has about 70-80 devices.

I have not seen any compatibility issues yet, the NIC that would feed signal to OPNSense is a 2 port 10GB HP NC550SFP which is a bit old, granted, but from what I have read, very stable. Just got it the other day.

[Vilhonator]

Oh in that case you won't have any issues other than maybe in speed when all devices are active and using all the bandwidth available, you can install iPerf and test that, but I doubt 70 to 80 devices all with 1Gb connection would overwhelm things especially if you have powerfull enough switch handling local network connections.

But back to the topic.

You can use any switch and AP with OpnSense. If you want to have minimum amount of management, then Ubiquity might be suitable, since all their APs and Switches use Ubiquitys own remote management software, which also can be installed on virtual machines, computer or you can purchase Cloudkey for it.

For 70-80 devices, well sadly I can only recommend Cisco Catalyst series switches (only Brand which I studied at school and got familiar with very well) which has 10Gb ports, so quite possibly very expensive, since Cisco doesn't have many managed switches with 10Gb ports that are exactly cheap.

When it comes to AP, all you have to check, is that if you can setup wifi password for AP on itself or does it require to be connected to a network device, which handles wifi protection (don't exactly know the english word to describe it, but I think you know what I mean). Opnsense does support some  M.2 Wifi modules (ones which password and network names are assigned by OS), so that is also an option when it comes to wireless solutions.

[SecCon]

TBH I am not worried about 1GB or 10GB ports and transfer rates either.

I have one bottleneck that I am currently aware of: disk I/O. On SMB3 with RAID5 or RAID6 that is at best 110MB/s. My ISP provides 100/100 MBit.  So I am kinda leaning on having that despite being able to throttle up to 10GBit/s on the OPNSense dedicated NIC. An average of 1GBit/s is enough for just about anything internally and the capacity I have on most device NIC's. Except for the printer, of course...   

Not chasing speed.

I might have to look at PoE though, so eventually one 48P "Core Switch" with that.

Did run Ubiquiti for about two years, long enough to learn that those guys prioritize hardware sales above application security and development. Sold my Edge router some time ago.

As for 70-80 devices, that's only about 30 wired, the rest is WiFi.   

[Vilhonator]

Yea, Ubiqity isn't exactly cheap when you are looking for quallity and security.

I worked at school and we used completely Ubiqity stuff there, amaizing performance yes, but we used about 5000€ (and that was tax free) from the budget to those things 

Only reasons why I would use Ubiqity at home, would be the possibility to have 1 webgui for firewall and 1 for switches and APs and (obviously) design. Ubiqity might be damn expensive to use, but at least I can't much complain about the design of the devices 

If you don't care about speed, then I can recommend to check CiscoCatalyst 1000 series switches, for example CiscoCatalyst 1000-48P-4G-L is around 1 100 € +  VAT around here, not that bad, though if you buy some of the 8 port versions, you might need more than one, never really had problem with my SG-350-8-P, but then again I only have NAS and my 1 PC with 1Gb NICs connected to it, so I don't really need better one untill I upgrade my NAS and switch to 10Gb networking.

You can also check Cisco APs (just stay away from Meraki, those require you to register to Cisco and you won't be able to manage them without internet connection).

But yea, Cisco SG or Catalyst and Mikrotik would be things I would look into, those are quite well worth the money to be honest. Never used Mikrotik myself, but I have heard it is brand worth checking.

I know they cost quite a bit, but well worth every penny. I have seen places using 20+ year old Cisco switches and heard they only had to shut them down once (outside scheduled reboots), and that was when they installed extra modules to them. Unless you live somewhere hot as heck, I doubt you would have any issues with Cisco devices, as long as you just invest on the right model.

[SecCon]

Well if you read my first post you will see I already have a Mikrotik Switch.

I sold my Mikrotik Router. My Ubiquiti Edge Router is sold. I have one UI Edge switch for sale with an AP. Not sold yet.

It would seem I may be best off for now keeping my Mikrotik 24P Switch, at least until I select a brand for both a PoE switch and an AP down the road.  Or maybe I should just reconnect my UI Edge stuff again and manage it via UISP: https://uisp.ui.com/. UI Edge UISP is not Ubiquity Unifi.

Bottom line seems to be that it does not matter a lot what hardware i combine my OPNSense with, as long as it is well known brands I should be ok, and that I will have to suffer at least one more MGMT interface in addition to OPNSense. :-(

[Vilhonator]

Well technically speaking, you can assing all 3 VLANs on single port on OpnSense, then create VLANs on your switch and rest of the devices, assign trunk port on the switch and access ports for each VLAN and connect trunk port to Opnsense, that way you only need 1 port for VLANs on your switch and devices connected to ports which have Access to management VLAN can access management.

Way I have done that, is that I have 2 VLANs on my LAN 1 port and assinged ports 8 9 and 10 combo ethernet SFP ports on my switch for VLAN trunk and ports 1-4 for VLAN 10 and 5-7 for management VLAN.

Yes, you do need to sacrifice at least 2 ports for VLANs (1 for active 1 for STP and failover switch in case either switch goes offline or cable gets bad etc. which is why many 16+ port switches have 4 SFP ports placed bit further from others forming a square)

But if you allready have 24 port switch and things are working just fine, then I would recommend just monitoring the network and thinking of matters as single point of failures bit more in depth and consider thinking about prepare to fix single point of failures (like extra switch just in case other has to reboot or fails) and maybe adding WIFI AP closer to devices which need wireless connections.

I mean, if you don't have one allready, and you are running a business, then UPS also might be worth considering before you get any extra devices (not that it could potentially be drastic, but more devices means more power consumption which does add up in electirc bills and puts some stress on fuses)

[Vilhonator]

But anyways, it doesn't matter what network device you use, opnsense is completely open source, so most compatibility issues comes in form of hardware and features, but if FreeBSD supports all the hardware machine has, then they also work on Opnsense.

In your case, I would try to figure how many single points of failures there are and how to fix them rather than thinking do I need better network gear.

If all works just fine as it is now, you can just add couple of extra devices of same models to your network.

[SecCon]

Single point of failure is power.

After that not sure really... it's not like I can setup a whole lot of redundancy for a home office. I do lack an UPS.

Got backups running on an independent server, spare cabling in case of rats, additional lan ports and cabling if any fries. But having spareparts is not the same as remediating SPoF.

[Vilhonator]

Well depending on power consumption, UPS will prevent things like devices just shutting down when power goes out.

Other single point of failure is if 1 switch goes off or 1 cable on specific port goes bad, then whole network is down, way to fix that is to add another switch, connect all devices to it and use STP (Spanning tree protocol), you will need to impliment it so, that if 1 switch goes completely out, the other will keep at least local network up and running.

Single point of failure in matter of networking, is basically anything that when failing, will cause whole network going down or part of the network.

Way I would plan this, is to get 1 extra switch to which connect all my wired devices and connect the two switches to eachothers VLAN trunk ports using 2 ethernet cables and setup spanning tree if it would require manual configuration. That way if cable to Opnsense gets bad, then internet is gone but I still manage everything locally (well except for OpnSense), if either 1 of the cables between the 2 switches dies out, then there's only going to be 1-10 second timeout till STP kicks in and internet etc. works again.

Obviously you can just connect both switches to each other and OpnSense to avoid internet going down as well, I tested STP with my switch connecting it to 2 ports on my Opnsense and it works quite well, sometimes I have to restart discord though, but not that big deal.

https://docs.opnsense.org/manual/multiwan.html#configuration gives you an idea what I mean, but yea.....biggest benefit it has, is being able to reboot and upgrade firmware on devices 1 by 1 etc. without loosing network connections

Opnsenses multi wan is just bit more advanced and beyond my level of skills and even not an option for me.

Oh just think carefully, because this type of thing is something you have to think from cost vs benefit point of view.
If so far, most connection issues you have had have been something, which having extra switch or firewall wouldn't have solved or network going down only means you have to replace cable or two, then having extra switch and AP is only something you can brag about having and learn / test new things.

[Vilhonator]

And I can't stress that enough.

Yes, single point of failure is scary thought, but your computer has at least 3 single points of failures (power supply, boot drive and motherboard), now how often do you have to replace those?

Networking stuff follows pretty much the same princible, but you have to consider cost even more carefully, because like any stuff you're interested, networking can get out of hand and you end up spending quite a bit. If you are too cheap, you end up replacing stuff often, if you spend too much, then CEO of your electricity comppany can start drooling everytime (s)he notices your power consumption.

What is good about switches, routers and firewalls, is that good UPS might be able to keep them up and running at least for next 5 hours without main power being on, so you can cut power costs by scheduling UPS to use battery at night untill you are awake and so on.

If you don't see potential costs being worth the investement, then I would say you can manage what you currently have and just save money for next BIG upgrade.

For example point where I would absolutely go for adding new stuff, is when there's leftover budget for it and I can't come up with anything else to spend it.