Hi All,
I'm new to OPNSense and would like to dip my toe in without spending much money. My eventual goal is to use OPNSense for my home network, replacing an older Fortinet N30. There's just two people using the network doing standard streaming, work from home, etc. and probably a dozen or so devices, wired and wireless. (I already have WiFi; just looking to replace the gateway.) So not very heavy network load, but would like to VPN in and play with and learn the various OPNSense modules.
Today and tomorrow are Amazon Prime Days. When I search "opnsense" they show a few mini PCs with dual NIC discounted about 30%. For example:
Beelink EQR6 Ai Mini PC, AMD Ryzen 5 6600H(8C/16T, Up to 4.5GHz), Mini Computer 16G DDR5 RAM 500GB NVMe PCIE4.0 SSD, Copilot Micro PC 4K@60Hz Dual Display HDMI/WiFi6/BT5.2/1000Mbps/W11 Pro https://www.amazon.com/gp/product/B09K39RJDQ/ref=ox_sc_act_title_1?smid=A1U8KYR6GMVLRX&th=1 (https://www.amazon.com/gp/product/B09K39RJDQ/ref=ox_sc_act_title_1?smid=A1U8KYR6GMVLRX&th=1)
Is this reasonable hardware to run a home firewall?
Or should I be shopping AliExpress, for example:
2024 pfSense Firewall Soft Router N100 N5105 N4000 4xIntel i226 2.5G LAN 2xDDR4 NVMe Fanless Mini PC HDMI2.0 DP AES-NI OPNsense
https://www.aliexpress.us/item/3256807066615315.html?src=google&pdp_npi=4%40dis%21USD%21164.81%2192.29%21%21%21%21%211%40%2112000039954707746%21ppc%21%21%21&src=google&albch=shopping&acnt=708-803-3821&isdl=y&slnk=&plac=&mtctp=&albbt=Google_7_shopping&aff_platform=google&aff_short_key=UneMJZVf&gclsrc=aw.ds&albagn=888888&ds_e_adid=&ds_e_matchtype=&ds_e_device=c&ds_e_network=x&ds_e_product_group_id=&ds_e_product_id=en3256807066615315&ds_e_product_merchant_id=106450275&ds_e_product_country=US&ds_e_product_language=en&ds_e_product_channel=online&ds_e_product_store_id=&ds_url_v=2&albcp=19678427463&albag=&isSmbAutoCall=false&needSmbHouyi=false&gad_source=1&gbraid=0AAAAAD6I-hFf8jXSWWgdtnnn-lsvPoC-1&gclid=Cj0KCQjwsJO4BhDoARIsADDv4vCQyHF2WXN6lWYQMkrdeP3nJDFbbDWUpla5hBl1qhkHGr8tA2YEg54aAodJEALw_wcB&gatewayAdapt=glo2usa (https://www.aliexpress.us/item/3256807066615315.html?src=google&pdp_npi=4%40dis%21USD%21164.81%2192.29%21%21%21%21%211%40%2112000039954707746%21ppc%21%21%21&src=google&albch=shopping&acnt=708-803-3821&isdl=y&slnk=&plac=&mtctp=&albbt=Google_7_shopping&aff_platform=google&aff_short_key=UneMJZVf&gclsrc=aw.ds&albagn=888888&ds_e_adid=&ds_e_matchtype=&ds_e_device=c&ds_e_network=x&ds_e_product_group_id=&ds_e_product_id=en3256807066615315&ds_e_product_merchant_id=106450275&ds_e_product_country=US&ds_e_product_language=en&ds_e_product_channel=online&ds_e_product_store_id=&ds_url_v=2&albcp=19678427463&albag=&isSmbAutoCall=false&needSmbHouyi=false&gad_source=1&gbraid=0AAAAAD6I-hFf8jXSWWgdtnnn-lsvPoC-1&gclid=Cj0KCQjwsJO4BhDoARIsADDv4vCQyHF2WXN6lWYQMkrdeP3nJDFbbDWUpla5hBl1qhkHGr8tA2YEg54aAodJEALw_wcB&gatewayAdapt=glo2usa)
Any help on finding a cost-efficient way to start using OPNSense will be much appreciated!
Thank you!
I have not looked at your proposals, but do not buy devices with REALTEK NICs. Many of the cheap offerings have those.
Quote from: meyergru on October 09, 2024, 09:28:00 AM
...do not buy devices with REALTEK NICs...
Thank you!
This excludes the Amazon option. The AliExpress option has 4x Intel i226 which I understand is supported FreeBSD/OPNSense.
Should one be concerned about the security of a BIOS provided by an unknown manufacturer? (Topton)
Is this even a reasonable question?
Thank you again!
Sent from my iPhone using Tapatalk
The support will not be too good, but if the firmware works for OpnSense, that should not be a problem. Microcode updates can be done separately.
As far as security goes, that is anyone's guess. I doubt that there is something fishy in there that goes beyond what Intel has been forced to include in it in the first place. Usually, these are just customized AMI bioses.
Okay! I will keep researching options from Topton and similar vendors.
Thank you again [mention]meyergru [/mention]!
Sent from my iPhone using Tapatalk
Topton or CWWK from Ali are decent options.
Check out Patrick's reviews on ServetheHome YT channel. Also on ServetheHome forums you can find threads and sometimes new BIOS-es. Also there's some work done to have coreboot running on some of these devices.
A level up from Ali would be Protectli devices. More expensive, coreboot and AMI bios-es with no updates in more than a year, and older hardware.
Another level up - Deciso devices such as DEC750 or DEC850. Check out the official recommendations for HW vendors here, especially if living in Europe.
https://forum.opnsense.org/index.php?PHPSESSID=e14n37bbk0bg8maqkjpl88lo1d&topic=2200.0 (https://forum.opnsense.org/index.php?PHPSESSID=e14n37bbk0bg8maqkjpl88lo1d&topic=2200.0)
There's also a DYI level, where anything goes from old HW to microATX boards to Supermicro servers - probably overkill fro most homes.
Biggest thing to worry about - as already mentioned - network cards. For future proofing your home go for 2.5Gb ethernet from the start.
Hello [mention]newsense [/mention]!
This is super helpful — thank you very much!
I had heard of coreboot, but really didn't know much about it. After doing a little research, coreboot will definitely be a requirement for my build.
Likewise, I really like the Protectli products. Not much more expensive than comparable Ali products (maybe $50?) but includes coreboot out of the box and US based support. So I'm pretty sure I will go with them.
I just need to decide 2 or 4 ports. I have VLAN capable switches downstream, so 2 ports could be enough. But the 4 port V series comes with 8 rather than 4 gigs of ram and more physical ports is always good, right? (Multi-WAN? separate physical network for WiFi? other?)
Anyways, this gives me a good start.
Thank you again!
Okay, so I watched a bunch of servethehome videos and now I'm particularly interested in the Qotom Q20331G9. (https://youtu.be/AKUTzjA1grE?si=3wLSoO0jk99JTy4x)
The Qotom is more expensive than the Protectli V1410. And with the Qotom, I would give up coreboot and live with the stock AMI BIOS. (After doing further reading, I was probably over-concerned about rogue BIOS anyways.) But in every other regard, the Qotom is far superior/future-proof. Especially attractive to me is the Atom 8 core processor and more expansion for memory and storage.
There is just one niggle...
Part of my justification for spending more on the Qotom would be to also build a NAS server on the same system, which implies virtualization to keep the edge firewall isolated from the internal NAS. But I've seen some comments that putting OPNSense in a VM is not a good idea due to lower I/O on the virtualization stack. Is this true? Is it not possible to "hardwire" a few NICs to a VM to bypass the virtualization overhead?
Or would it be wiser to just keep it simple and purchase a less capable device (eg Protectli V1410) and run only OPNSense bare metal?
Thanks in advance for any comments or suggestions!
Sent from my iPhone using Tapatalk
I will always advocate for a dedicated box doing security stuff at the edge. By virtualizing you trade some of the protections you'd have running it bare metal and rely on whatever the hypervisor provides. No hypervisor is designed to be internet facing otherwise there would be no more physical firewalls to speak of in decades. This is not to say you cannot or shouldn't run a virtualized FW on your hypervisor, even if behind a physical one.
From Protectli you have either the V1410 with a more capable N5105 CPU or the FW4C. As long as you have 8GB ram you can't go wrong with either of them.
Quote from: newsense on October 14, 2024, 05:27:28 PM
I will always advocate for a dedicated box doing security stuff at the edge
...
No hypervisor is designed to be internet facing
...
otherwise there would be no more physical firewalls to speak of in decades...
Hi again @newsense. Thank you for your inputs!
I agree the common/proved wisdom is to run edge security bare metal on it's own box. Another benefit not mentioned yet is decoupling maintenance cycles on the edge and the VM platform. It's nice to have the internet up as you're patching your VM platform.
...
I don't agree that hypervisors should not run public-facing hosts. In fact, I think it is common/best-practice to run most/all public facing hosts (eg. mail, web, etc.) virtualized as the security and non-security benefits outweigh the risks, which can be managed.
...
I believe the main reason for the persistence of physical firewalls is to maintain consistent performance without fear of resource contention and with relative freedom to develop complicated rulesets, DPI, VPN, etc. My experience managing networks with many hundreds of clients and many tens of thousands of connections using software-based firewalls (Checkpoint, Sonicwall) versus ASIC-based firewalls has imprinted me for life. :'(
Fortunately, this is not my use-case here. I'm only building a residential gateway for 2-4 people and approximately zero public facing services. (A future camera server behind a VPN is the only exception I can think of ATM.) My original concern was that possibly the overhead from virtualized I/O might prove too costly to maintain throughput on a small, fanless device. But since then I've read more about Proxmox CPU and NIC pinning, and I believe one can work around virtualized I/O overhead.
That said, I think it probably
is wiser to run edge security/routing on bare metal, mostly to keep Internet access disconnected from internal maintenance events as mentioned above. So I will probably head that way and Protectly is still the leading option. And yeah, running a second internal firewall on a VM might be fun. I may look at that in the future.
Thanks again!
I would certainly agree with the "internet while patching" statement.
One thing that I'd have to figure out if I virtualized my firewall is how to get the single connection to the firewall when the firewall gets moved from host to host during patching? And also spoof the MAC while I'm doing all that?
Using XCP-NG the normal procedure for updates is to do a "rolling pool" update. Click the button and it goes through and migrates the VMs to other hosts, updates the "control host", then moves VMs back and continues doing this down the stack until all the hosts are updated.
I guess you could maybe set it as an HA system and just run one instance on each host, I'd have to think it through.
I'm firmly in the bare metal camp as a possible way to prevent vlan hopping or other things on a shared connection, but a lot of places virtualize their firewalls, and that includes the more modern Cisco stuff that my work runs.
Quote from: Greg_E on October 16, 2024, 03:48:44 PM
... how to get the single connection to the firewall when the firewall gets moved from host to host during patching? And also spoof the MAC while I'm doing all that?
Hi Greg!
Ouch! The more we talk about this, the more I'm convinced to go bare metal!
Just waiting for the Protectli v1410 to come off back order, or to see if maybe the v1610 is going to be released soon and if it is reasonably priced. I don't think I will need the extra ports, or even the faster processor. But I'm a little concerned about being tied to 8G ram forever. I don't have any experience with OPNSense. On the other hand, I've never regretted having more ram, or being able to upgrade it later.
Related, the soldered on ram concern is making the Topton N5105, 4x i226v box at $120 w/o memory or storage (https://www.aliexpress.us/item/3256806241034001.html?spm=a2g0o.productlist.main.1.7733af5fSlbDF4&algo_pvid=595149f7-d15c-4f79-941e-bfac78b91dec&algo_exp_id=595149f7-d15c-4f79-941e-bfac78b91dec-0&pdp_npi=4%40dis%21USD%21234.82%21136.20%21%21%21234.82%21136.20%21%402103247017291365312158922e85e1%2112000038882385461%21sea%21US%216152741746%21X&curPageLogUid=59YgnfuzTdv2&utparam-url=scene%3Asearch%7Cquery_from%3A) pretty tempting. I would like to support Protectli and benefit from their work and that community. But the cost delta is pretty significant, especially because I have a stick of DDR4 and some old 2.5" SATA SSDs on hand, which the Topton can use...
Too many choices! :)
Maybe this is unhelpful but for another consideration. I have been running it virtualised for a while. The previous APU is backup and that ran it baremetal.
It is a CWWK. Amd processor with Intel nics. Works absolutely fine and I don't much get the argument of not putting it on a VM. See the usage
Does it make it a bit more complicated?, sure. To me the pros overweight the cons, of which of course there are.
I'm suggesting 16gb of RAM because my use case shows that sometimes I am using 8+, not much over 8, but still over. Normally hovers around 6.5-7GB in use.
If you virtualize it, do you lock it to a single host or can it move?
If it moves do you set up a separate "switch" for the WAN connections and use dedicated (or vlan) for all the hosts? I guess this is where I would start if I were going to virtualize it. Still concerned about vlan hopping attacks, but other systems are fine with putting their firewall on a VM. And maybe I should work this out for myself one of these days.
Hi. Guess you're asking me :)
Definitively the RAM amount depends on usage, what services are enabled mostly of course. My APU has 4 Gb and an low passmark. Worked fine for everything up to 1 Gb ISP package until I enabled Zenarmor. Then it would just not be able to cope. So I upgaded the machine.
I did find to my pleasant surprise that this AMD cpu needs only 2 vcpus for all services I ran including Suricata and Zenarmor. I did increase the RAM to 6 GB but now Suricata is disabled. I could probably reduce the RAM again.
Honestly these AMD cpus are so amazing that I fail to see why so many people keep looking for Intels. But everyone has a preference so here we are.
> If you virtualize it, do you lock it to a single host or can it move?
No locks. It can move although I haven't had to prove it. To my knowledge these CPUs don't have vendor lock as they do (or did) from Lenovo.
Oh wait I see what you mean, the VM.
This is a router/firewall for my home only, not an enterprise so I don't need to move it for maintenance. Just schedule downtime, take backups, upgrade, reboot VM. All simple.
If maintenance of the host, pretty much the same.
A few weeks back I had my OS SSD die without warning. Cheap SSD from Amazon on offer. Brand from the PRC that floods Amazon. Anyways, I save the VM config files from time to time, so it tested my Disaster Recovery. Some improvements to make but truly so much more power-efficient and flexible with a VM.
VLAN hopping attacks. Well, if someone is hopping VLANs on my network, I have bigger problems already, so no that doesn't keep me up at night.
Hi All,
I eventually did purchase a system and wanted to share with everyone what I did and how it turned out.
I was and still am a fan of Protectli, but I ended up not going with them. I really wanted 16G for "future proofing" the system, and they didn't have this available in a system at a good price for me. Instead, I went with a Topton N5105 (https://www.aliexpress.us/item/3256806241034001.html?spm=a2g0o.order_list.order_list_main.11.5e201802KVL2by&gatewayAdapt=glo2usa) with (4) i226 which I got with no memory or storage for $115 on a "special". I chose the N5105 because of the price and because I read that while the N100 had higher peak performance, the N5105 was on par for steady loads, at least within a given thermal envelope. Also, I had a 4G DDR4 stick laying around to test with the N5105, along with a small SSD to test with.
Once I confirmed the system would boot and behave normally, I purchased 2x8G=16G Crucial ram and a 250G Kingston nvme disk for less than $70. I installed these components and reinstalled the system from USB. I then ran the system on my bench for about a week, in and out of production, to see how the hardware performed and how OPNSense worked for my environment. (My first time running OPNSense.) My only concern was the temperature of the heat sink, which I knew would get warmer once I installed it in the closet, where all of my network gear lives. So I purchased a USB powered fan (https://www.amazon.com/gp/product/B06Y5WWBHH/ref=ppx_yo_dt_b_asin_title_o00_s00?ie=UTF8&th=1) for less than $10. This fan has multiple speeds and happens to fit the form factor of the Topton perfectly. I removed the grate from one side and attached it to the Topton with double sided tape (also held in place by the wall mount.) On the low speed and when sitting on my work bench, the fan lowered the temp at least 5 degrees celsius. Once I installed it in the closet where the heat builds up more, the fan (still on low speed) lowered the temperature from over 70 to about 60 celsius. So I think getting the fan was a good choice.
My total cost is right around $200 and I've been very happy with the hardware and with OPNSense thus far. My initial motivation was to separate WLAN IOT devices from my home LAN (computers), and that is now accomplished. I bridged two of the ports for my APs (to avoid having to purchase another switch) and this was a little finicky to set up. But I eventually figured it out and it all seems to be working now. I'm using less than 10% of memory and less than 1% of disk, so I can easily grow the system going forward.
Still to do or may do:
- investigate why my ChatGPT client now loses its connection more frequently, which seems to have started with use of OPNSense
- review logs and improve firewall rules, spend some time with Unbound DNS
- consider running ram disks for /tmp and logging (would want to sync logs to disk periodically)
- consider security plugins like Suricata, Zenarmor, others?
- consider use cases for squid or other caching
- consider use cases for Tor or VPNs
- reinstall on Proxmox to run both OPNSense and a Ubiquiti Unifi controller in a Docker container
Special thanks to @newsense, @cookiemonster, @meyergru and everyone else who chimed in.
Cheers!
-J