Hi all,
I have been running opnsense for almost two years now on a fujitsu futro S920 with 8GB ram and AMD GX-222GC SOC CPU. I know this machine is not the strongest out there but it serve me well on my previous connection which was 200MB/50MB (download/upload). In that setup I was also running Openvpn, wireguard, suricata on wan in ids mode and zenarmor in LAN in ips mode. I have arround 50-60 devices connected to the internet (but most of them are IOT devices). Ok things were not ideal due to one of the nics being a realtek but still I was happy giving the amount of money put to it.
Now I have upgrade to a fiber connection of 1GB/250MB (download/upload) speed. In order to get the most of my router I replaced zenarmor with adguard and make some tweaks on the tunnables of the router. Overall I do not see the cpu gets bottleneck all the time but when I speedtest (from a wired pc directly connected to the router) I only get in the best case scenario ~850MB download. Most of the times my speed is capped at around ~550MB. Not sure if there is something I can do more to get more of my speed, I tried disabling suricata and stopping other services but the result was the same.
So I am thinking to moving to new hardware and migrating everything to a new router. I search online to either a dell/HP/lenovo SFF pc or either a ready made router from aliexpress (with N150 cpu and 16GB ram) but I having trouble figuring out whether the new system will be enough.
My requirements are:
1) Being able to get my full speed 1GB/250MB
2) Run OpenVPN for 2-3 clients (not heavy traffic all the time)
3) Run wireguard for 2-3 clients (not heavy traffic all the time)
4) Have a few VLans configured
5) Enable IPv6 in the near future
and ideally ...
6) Run Suricata in IPS mode in wan
7) Run Zenarmor in IPS mode in LAN
Is the N150 even close enough to what I want to achieve or I need to stay clear? What is the recommended hardware for my setup? What are your thoughts on the matter?
Thanks
Phanos
N150 can do it.
It usually boils down to OPNsense stuff.
Read my N150 post (https://forum.opnsense.org/index.php?topic=48166.0), note the hardware being used, three 2.5G copper and two 10G sfp. I run Suricata IPS mode, it's the resource hog. Look for similar hardware. Load testing (LAN clients accessing internet via WAN, etc) is always key when it come to performance. IPS, IPsec, Proxy, plugins, etc etc.
My mem usage is very low, disk usage about nill. 16GB ram 512GB ssd (nvme, etc) seems good. If you can squeeze in 32BG ram that's good too. Choose hardware that can run the fastest RAM, etc.
That device I got has a low noise fan, I saw temps from Lobby saying it got near 61C, N150 has max op temp of about 110C. I will add two small 40mm fans to the bottom plate in push-pull orientation (i'll 3d print a thin cradle for the device to sit in, etc). To keep fans quiet (albeit lowering cfm) I run 24v fans on 12v power.
Above 2.5G the LAN side switching then becomes another look-at point. Can have fast on WAN side, but the LAN side needs it too.
thanks BrandyWine for the info. are you running zenarmor on lan as well or just suricata on the wan? how much tweak did you perform on the opnsense side after installing?
I also thinking of going with a cpu i3-1215u instead of the N150 but not sure if it is worth it.
No zenarmor yet. Suricata on wan is the ET Pro Telemetry (free) plugin.
I tweaked just a few items using Tunables, nothing crazy. See "Built on N150" thread.
i3 is in every metric way better than the N150. You are comparing different classes of cpu though.
https://www.cpu-monkey.com/en/compare_cpu-intel_processor_n150-vs-intel_core_i3_1215u
i3, 32GB ram, 512G or 1T NVMe, makes for an OPNsense device that will last a long time.
Be sure to seek out hardware that has fastest mem controller.
And from what I can see, the better i3 will run near +$60-100(US) over N150 device.
In % though, getting i3 is approx 50% more money.
I end up buying a pentium 8505 with 16GB ram and 4 2.5GB network ports. I think running zenarmor will still not be enough and get the full 1GB speed of my ISP but the alternative options were too expensive anyway. Will try of course the setup and see how it goes when it arrives.
8505 is substantially better than N150. i3 is better, perhaps due to it's bigger cache in all levels.
8505 w/ 16GB RAM and some decent SSD, should be ok.
Hi! new member here! Sorry for using this thread but I am just pulling the trigger on a Topton 2x10Gbps SFP + 3x2,5Gbps eth i-226 N150 8GB DDR5 128GB Nvme.
I wonder if 8GB DDR5 is just enough for home use. I plan to use it same way the original post of this thread.
thanks!
Yes, it should do just fine. Heed the N1x0 warnings in #23 here (https://forum.opnsense.org/index.php?topic=42985.0). Also, disable ASPM in BIOS or set hw.pci.enable_aspm=0 in tuneables. Both I226 and 82599ES expose problems when ASPM is enabled.
Hello, I want to build a new low power OPNsense firewall(as Proxmox VM) and consider Odroid H4 + Netcard2(4x i226).
Is it a good candidate for low idle power consumption (7-8w idle max)?
Odroid H4 has good ASPM support but here I found the warning about Intel i226 related problems with ASPM enabled.
Does the ASPM-enabled problem also affecting virtual OPNsense installations @Proxmox with VTNET virtual adaptors?
Can I use the Netcard board 4-ports for virtual adaptors in proxmox?
Thanks in advance for advice
I do not believe that this goal is achievable. The H4 with the N97 CPU has a CPU TDP of 12W, which is worse than the N100 at 6W, while the N305 has 15W. Of course, at idle, the CPU draws less power, but really, those numbers are overoptimistic - I see my N100 draw 25W at peak load.
Also, You need ~2W for the chipset, at least 1W for RAM, 0.5-1W for each NIC, so you will be at least at ~10W. Considering efficiency of your power supply, you will end up with more like 12-15W at the wall.
All of that is not even considering using OpnSense as a VM on Proxmox, which also has background tasks running.
What do you consider "good ASPM support"? Does the BIOS allow for disabling it? If not, you may get into trouble, because on these platforms, FreeBSD does not support it, so you will have network freezes, see this: https://forum.opnsense.org/index.php?topic=48562
I neither understand what is the meaning of "good ASPM support"?
When you are researching HW for OPNsense, do not look what kind of ASPN support device has, more or less look for if it can be disabled. Taking ASPM support into the equation of which HW to buy sounds to me pointless.
If you want a future proof device you have two choices currently
1. Official OPNsense DEC HW
2. Mini PC N100 and above (N100 is more than enough)
Regards,
S.
OK, sorry, I'm not native speaker and that phrase was too simple. By "Good ASPM support" I mean:
1. Firmware/bios is tuned so power saving modes are working properly with all hardware components integrated onboard
2. Enabling ASPM causes significant reduction of power consumption in idle
There are not many companies who take power saving seriously but Hardkernel is one of the exceptions.
You can google several H3/H4-related fine-tuned builds online which can go down to around 3-5 watts idle (in Linux):
https://www.hardkernel.com/shop/odroid-h4/
https://forums.unraid.net/topic/167669-odroid-h4-intel-n97-2x25gbit-4x-sata-1x-m2-ddr5-max-48gb-with-ecc/
I know OPNsense is not based on Linux but Proxmox is. That's why I want to try a build with H4@Proxmox with it's linux drivers.
Maybe it's completely stupid idea, VM-related performance/power penalty will be too high and I end-up with ASPM-disabled physical server installation like most of you recommend... but I'd like to try and test :)
You can set cpu power states without using powerd or aspm.
Search for my "Built on N150" thread, in there are some tunables for cpu power states.
I not sure why anything else in the system should be powered down (sleep, etc), it's a fw device.
Hello, I've been researching this topic thoroughly for a while now and so far it's been very hard for me to find sane, affordable hardware options with the requirements that I have.
My requirements are the following:
- 6x RJ45 1Gb/s Ports (2.5Gb/s is nice to have but not required)
- Active cooling but quiet when idle (I don't trust passively cooled devices that run 24/7 very much and thermal wear on other components is much lower)
- Proper device monitoring including mainboard temperatures and fan PWM
- Hardware properly supported by FreeBSD
- Low power draw
- Affordable, meaning the hardware costs around 500€ (a bit more is also fine)
I don't mind buying passively cooled hardware and upgrade it with a fan myself, as long as it fulfills the requirements posted above. Please note that I don't want to deal with external USB fans, the fans should be installed inside the device.
So far I have checked a lot of the N100/N150 desktop mini PC hardware, however most of them contain ITE IT8613E Super I/O chips, meaning proper monitoring is not possible. A hwmon driver has been written for this Super I/O chip, however it has been in review for 2+ years (https://reviews.freebsd.org/D39970) and who knows when it will be finished. I wouldn't mind helping out with that but just reading up on all the bureaucratic procedures regarding this is causing me headaches already. All the posted warnings about very specific N100/N150 incompatibilities, are making me wary of this option as well - I don't want to buy hardware and then see random errors piling up later, that can't possibly be fixed due to BIOS constraints and the like.
On the positive side: I've tried out such an appliance before and I was able to tweak the processor options in the BIOS and OPNsense similar to what has been stated in the "Built on N150" thread. It helped to reduce power consumption, as well as heat by a good margin.
Now, I've done some research and looked at my options:
- Official OPNsense hardware: No affordable device with 6x RJ45 ports (costs over 1,500€).
- Landitec: Business to business only.
- Thomas-Krenn AG: Same as Landitec.
- Known desktop brands (ASUS, Intel, etc.): Haven't been able to find any device with 6x RJ45 ports, not even taking low power draw into consideration.
- CWWK: There is a whole zoo of products that ultimately are only rebranded CWWK hardware. Contains the aforementioned ITE IT8613E chips which makes hardware monitoring impossible, needs manual fan installation and the quality assurance process is questionable (I've heard about "magic smoke" coming out of the power supply in some cases). I'd rather buy from a more reputable manufacturer.
- Protectli: Same as CWWK, except the brand is more reputable and therefor the quality assurance is probably a lot better. Same hardware monitoring problem and I'm not sure if you can manually install a fan in all of them.
- NRG Systems: Has a N150 rack option (IPU610) with three fans, which is more in line with what I'm thinking about but the case fans can only run on full power with no PWM input, meaning quite an amount of hardware tweaking is required to make them work the way I like to. Sadly contains a ITE IT8613E chip as well and I don't have a rack (though rack installation is not necessarily required but will require buying a larger open shelf).
Can someone more knowledgeable than me maybe provide some insight or assistance? I don't have CPU heavy workloads, meaning no VPN or packet inspection, having a bit of "leeway" with the CPU is a plus however, traffic shaping could be an interesting option.
It depends on which of your requirements you deem indispensable:
- low power (I guess)
- active cooling
- temperature monitoring
- future proof by having support of a supplier that has BIOS updates
- low price (i.e. under 500$)
I doubt that you will find something that ticks all boxes.
Also, apart from #1 low power and #5 low price, I find the other goals quite debatable:
1. Active cooling is only neccesary if the device draws too much power to be cooled passively. That is essentially not the case with N100/N150 boxes, if they are TDP limited and do not have more than 4 I226 NICs. Considering the price, you will have to invest in 3-4 years anyways, because then at least the SSD will be toast because of heavy writing, so thermal wear & tear is not a consideration.
2. Temperature monitoring is dispensable IMHO, because you can measure the CPU core temps and the SSD temperature. The former can even be display on the dashboard. The components are so close together that you can almost guess what the board temperature will be. Also: why would you want that in an appliance? It has to do its job, you do not have to watch its health while it does that.
3. Having a supplier who actually fixes problems with BIOS updates will contradict your price requirements (both Protectli and Deciso options cost more). Also, from my observations of the last 4 years, needed CPU microcode updates were provided directly in FreeBSD/OpnSense. The ASPM problems that led to instabilities now were there all the time and popped up just now. We know how to avoid them (turn off ASPM) and that is about it. Temporary difficulties with new OpnSense CE releases can always occur and are part of what you buy into when you do not buy (or did you pay for the community edition?). If you do not want that, get yourself a Business edition, which is more stable.
Let's not get sidetracked and refocus on the original problem again.
Hardware requirements:
- 6x RJ45 1Gb/s LAN ports
- Quiet, active cooling (or retrofitting a fan is possible)
- Hardware properly supported by FreeBSD
- Low power draw
Corrections:
- I need full hardware monitoring (temperatures, voltage, fan PWM), not just CPU/SSD temperatures
- BIOS updates are not on my list, they are optional but nice to have
- Cost is ideally ~500€ (like mentioned, can be stretched, e.g. to 700€), it is not "under 500$ (425€)"
Some clarifications for context, no need to debate:
- SSD endurance does not matter, even a cheap WD Blue (150 TBW) would need 137 GB of writes per day to hit that limit in 3 years. OPNsense logs are in the range of a few GB per day and anything bigger needs to go into a proper log aggregation system like Loki to be of use anyway.
- A single 80x80 mm intake 1W fan dropped the CPU temperature by 10°C on my test device, despite the CPU being installed on the opposite site of the casing. More headroom means less throttling, less power draw, higher performance and higher lifespan of nearly all components. And I like to have the option to use the performance I paid for.
- Proper Super I/O sensors help with detecting fan issues and system cooling issues before they become a problem. They detect degraded VRMs and may even detect a dying PSU, which can show up as voltage deviations. CPU and SSD temperature sensors are not related to this in any way and can not detect most of these issues reliably.
- Not monitoring a 24/7 edge device is the unusual position here, not the other way round. Every serious vendor exposes board sensors by default. Proper hardware monitoring helps to identify problems before they become actual hardware failures and allows shutdown options through self-monitoring where BIOS options are not available or reliable.
Questions that are still open:
- Turning ASPM off - Does this have any negative side effects aside from an increase in PCIe power consumption and is this bug present on every hardware?
- What is the minimum budget for the requirements I've set?
- Which hardware does ship with supported full hardware monitoring?
- Are there other vendors that offer OPNsense compatible hardware, aside from the ones I listed yet?
Quote from: UbiquitousWhite on September 06, 2025, 07:01:27 PMI need full hardware monitoring (temperatures, voltage, fan PWM), not just CPU/SSD temperatures
https://www.supermicro.com/en/products/motherboard/A2SDi-4C-HLN4F
Comes with IPMI, full monitoring of everything like voltages, fan speed, temperatures availably with e.g. Observium. Can easily drive a 1Gbit/s uplink, no experience with higher speeds.
Quote from: UbiquitousWhite on September 06, 2025, 07:01:27 PMSome clarifications for context, no need to debate:
- SSD endurance does not matter, even a cheap WD Blue (150 TBW) would need 137 GB of writes per day to hit that limit in 3 years. OPNsense logs are in the range of a few GB per day and anything bigger needs to go into a proper log aggregation system like Loki to be of use anyway.
O.K. - last bit of debate about just one error in your string beliefs. Here is the smartctl output of a lightly use home installation of OpnSense:
Quote# smartctl -a /dev/nvme0ns1
smartctl 7.5 2025-04-30 r5714 [FreeBSD 14.3-RELEASE-p2 amd64] (local build)
Copyright (C) 2002-25, Bruce Allen, Christian Franke, www.smartmontools.org
=== START OF INFORMATION SECTION ===
Model Number: KIOXIA-EXCERIA G2 SSD
Serial Number: XXXXXXXXXX
Firmware Version: ECFA17.3
PCI Vendor/Subsystem ID: 0x1e0f
IEEE OUI Identifier: 0x8ce38e
Total NVM Capacity: 500,107,862,016 [500 GB]
Unallocated NVM Capacity: 0
Controller ID: 1
NVMe Version: 1.3
Number of Namespaces: 1
Namespace 1 Size/Capacity: 500,107,862,016 [500 GB]
Namespace 1 Formatted LBA Size: 4096
Namespace 1 IEEE EUI-64: 8ce38e 0300993420
Local Time is: Sun Sep 7 12:49:56 2025 CEST
Firmware Updates (0x12): 1 Slot, no Reset required
Optional Admin Commands (0x0017): Security Format Frmw_DL Self_Test
Optional NVM Commands (0x005f): Comp Wr_Unc DS_Mngmt Wr_Zero Sav/Sel_Feat Timestmp
Log Page Attributes (0x0a): Cmd_Eff_Lg Telmtry_Lg
Maximum Data Transfer Size: 512 Pages
Warning Comp. Temp. Threshold: 72 Celsius
Critical Comp. Temp. Threshold: 90 Celsius
Supported Power States
St Op Max Active Idle RL RT WL WT Ent_Lat Ex_Lat
0 + 7.69W - - 0 0 0 0 1 1
1 + 6.18W - - 1 1 1 1 1 1
2 + 5.42W - - 2 2 2 2 1 1
3 - 0.0500W - - 3 3 3 3 7000 5000
4 - 0.0050W - - 4 4 4 4 13000 36000
Supported LBA Sizes (NSID 0x1)
Id Fmt Data Metadt Rel_Perf
0 - 512 0 2
1 + 4096 0 1
=== START OF SMART DATA SECTION ===
SMART overall-health self-assessment test result: PASSED
SMART/Health Information (NVMe Log 0x02, NSID 0xffffffff)
Critical Warning: 0x00
Temperature: 59 Celsius
Available Spare: 100%
Available Spare Threshold: 5%
Percentage Used: 14%
Data Units Read: 3,715,694 [1.90 TB]
Data Units Written: 38,131,588 [19.5 TB]
Host Read Commands: 82,686,672
Host Write Commands: 294,063,995
Controller Busy Time: 619
Power Cycles: 13
Power On Hours: 3,661
Unsafe Shutdowns: 5
Media and Data Integrity Errors: 0
Error Information Log Entries: 86
Warning Comp. Temperature Time: 0
Critical Comp. Temperature Time: 0
Thermal Temp. 1 Transition Count: 129
Thermal Temp. 1 Total Time: 22224
Error Information (NVMe Log 0x01, 16 of 63 entries)
No Errors Logged
Self-test Log (NVMe Log 0x06, NSID 0xffffffff)
Self-test status: No self-test in progress
No Self-tests Logged
Oh, just in case you do not have a calculator at hand, that amounts to 222 GByte/day, and BTW: those were
not logs. You'll learn the hard way.
Good luck, I am out of here.
Quote from: Patrick M. Hausen on September 07, 2025, 12:06:54 AMhttps://www.supermicro.com/en/products/motherboard/A2SDi-4C-HLN4F
Comes with IPMI, full monitoring of everything like voltages, fan speed, temperatures availably with e.g. Observium. Can easily drive a 1Gbit/s uplink, no experience with higher speeds.
Thanks for the recommendation! Indeed, it is starting to look like that I have to utilize IPMI and do a custom build. Not my favourite option but probably what I have to go for in the end.
Quote from: meyergru on September 07, 2025, 01:04:32 PMO.K. - last bit of debate about just one error in your string beliefs. Here is the smartctl output of a lightly use home installation of OpnSense:
[...]
Oh, just in case you do not have a calculator at hand, that amounts to 222 GByte/day, and BTW: those were not logs. You'll learn the hard way.
Good luck, I am out of here.
19.5 TB data written over 3,661 power on hours should amount to 128 GB per day - not 222 GB per day.
Regardless, SSD endurance is still not a problem:
- A 500 GB WD Red SN700 NVMe SSD (advertised for use in NAS environments) has an endurance of 1000 TBW and costs only marginally more than a budget WD NVMe SSD. Even at a rate of 222 GB data written per day, it would take 12 years for the SSD to exceed the guaranteed endurance.
- Plenty of people are running their OPNsense comfortably for many years on SSDs with much less endurance and without replacing them. It does not seem to be the limiting factor.
I'm not here to debate my requirements; I do not find it productive and they are not
that outlandish either. Of course I'm open to questions and very thankful for security concerns I have overlooked, such as the vendor BIOS updates but primarily I'm here to find out what options exist and whether I need to increase my budget.
If TBW is of any concern, I would use device that you can install two NVMe SSD's where the 2nd one is same size but low budget, and then every so often (6mo, 1yr) boot device with a liveLinux usb and ddrescue from primary to bkup SSD. When the primary dies you can image back.
Another idea....
I have just bought a topton miniPC from aliexpress, N150 with 3x i226-v and 2x 82599ES.
I added a tplink switch with 4x 2.5Gbps PoE + 1x 2.5Gbps non PoE and 1x 10Gbps SFP.
8x 2.5Gbps ports in total.
Both cost me USD 272,95
With a 500€ budget you can buy that setup twice. And have one as spare/stand by in case of any issue.