OPNsense Forum
English Forums => Hardware and Performance => Topic started by: TheWhiteBuffalo on August 12, 2018, 03:22:15 am
-
Greetings all,
I am currently running my firewall on a netbook and I am looking to get something with a smaller form factor. I have been looking at the Asus Tinker Board with the Rockchip RK3288 CPU (https://tinkerboarding.co.uk/wiki/index.php?title=Hardware#Specifications). My main question is: Has anyone had success running OPNsense on this board?
I hope this finds you all well,
TWB
UPDATE 8/14/2018:
The Tinker Board is out of the scope of options for me. However, I am still looking for a single board computer to use for my firewall.
Any suggestions on my best possible options would be greatly appreciated.
Thanks again,
TWB
-
You need to say what the budget is. :)
-
The Tinker Board is out of the scope of options for me. However, I am still looking for a single board computer to use for my firewall.
The problem is the availability of freebsd/hw drivers for the most sbc. Even if the cpu (i.e. ARM-Core) is supported by freebsd that doesn't mean that the whole sbc is running with freebsd!
So best solution at the moment is to use a x86-sbc for pfsense. There is a wide range of x86-sbc in the market but many of them where sold only to industrial customers.
One Company that sell x86-sbc is PC-Engines with their APU-Series (i.e. https://www.pcengines.ch/apu2.htm). IMHO the APU-Series is a good solution for a small and efficent firewall.
best regards
Dirk
-
The Tinker Board is out of the scope of options for me. However, I am still looking for a single board computer to use for my firewall.
The problem is the availability of freebsd/hw drivers for the most sbc. Even if the cpu (i.e. ARM-Core) is supported by freebsd that doesn't mean that the whole sbc is running with freebsd!
IMHO the APU-Series is a good solution for a small and efficent firewall.
best regards
Dirk
Beware PCengines APU2, if you want to use it on 1-Gigabit links, where it fails miserably to reach anything close to wire-speed, compared to some Linux-based firewall distrib.
https://forum.opnsense.org/index.php?topic=9264.0
-
@ricsip, honest question: does your answer meet standards for objectivity? a lot of users have no problem with apu2 performance or are reasonable about its limits and this seems to indicate a general "no go" for said hardware.
-
@ricsip, honest question: does your answer meet standards for objectivity? a lot of users have no problem with apu2 performance or are reasonable about its limits and this seems to indicate a general "no go" for said hardware.
@franco: with all respect, I stated clearly that it's the combination of "Gigabit speed and APU2" , that cannot be taken as granted. That's what I learned personally.
-
A yes or no would have been clearer. FWIW, you seem emotionally attached to the fact that this hardware doesn't live up to your expectations. Please don't let your sentiment lessen your appearance while helping others.
Cheers,
Franco
-
Awesome information! thank you for all the responses.
You need to say what the budget is. :)
As it stands, my budget for this project must not exceed 300 USD, however, I would like to spend much less than that.
The problem is the availability of freebsd/hw drivers for the most sbc. Even if the cpu (i.e. ARM-Core) is supported by freebsd that doesn't mean that the whole sbc is running with freebsd!
Since my original post, some personal research has revealed exactly this. Simply put, I need a x86-sbc to be successful with this project. I will definitely be checking out the APU-Series as a viable option. Any other suggestions are welcomed and would be greatly appreciated.
On a side note and, maybe, an unrelated tangent for a different discussion board all together. Is it worth trying to compile a custom freeBSD kernel to support some of these ARM boards? What would it take to create the drivers for different hardware?
I have done this with Linux and some obscure hardware setups, however, I am definitely a newbie to the BSD world.
Thanks again for the feedback,
TWB
-
Armv6 support which we tried (RPI2) is fragmented per device. armv7 is similar. armv64 promised to bring the devices closer together under a single kernel, but last I heard this is part of FreeBSD 12 and we're currently using 11.
While it's possible to build armv6/7 for a single device the effort is currently steep, but not impossible given the open source nature. Only trouble is getting the build power or cross-build up and running properly and maintaining this for all subsequent updates. Some users have reported success in these areas, but not to the extent that it was easily usable for others and provide a constant stream of updates.
We do not wish for vendor locks and thus hope for better days in FreeBSD 12 with a unified arm64 platform. Fingers crossed. :)
Cheers,
Franco
-
While it's possible to build armv6/7 for a single device the effort is currently steep, but not impossible given the open source nature.
I had a feeling this was the scenario. I suppose if it was easy we would already have support for these boards. I might still look into the process a bit more. For now, however, it looks like buying a board that is already compatible with the current build will be more time efficient.
Greatly appreciative for the information,
TWB
-
A yes or no would have been clearer. FWIW, you seem emotionally attached to the fact that this hardware doesn't live up to your expectations. Please don't let your sentiment lessen your appearance while helping others.
Cheers,
Franco
Yes, indeed, I am usually passionate about things I do.
Let me ask something. If someone recommends the APU2 to OP, what reason is behind that recommendation? The OP didnt clarify the bandwidth requirements. It does matter if the APU2 will be serving a 10 Mbit ADSL or a 1Gbit Fiber WAN with PPPoE. It does matter if there will be 2 lines in the firewall ruleset, or 1000. It does matter if IPS/IDS will be activated with many rules, or no IPS/IDS at all.
There are no publically available benchmarks, that could say, OPnsense 18.7 + APU2 is good up to X mbit traffic (with NAT / without NAT?) if adding the following layer, one after each other:
a) there are X inbound rules in pf
b) there are Y rules in Suricata
c) x sessions in openVPN using y crypto algorithm serving z mbit bandwidth
d) total aggregated CPU load is under 400%
So its neither sufficient to say in generic, that APU2 is what OP is looking for.
-
As it stands, my budget for this project must not exceed 300 USD, however, I would like to spend much less than that.
Go for a Qotom, if you have the ram and SSD get the barebones i5 version, if not 4Gb RAM and a 32Gb SSD will still only set you back $289
Rock solid units, lots of users here have them, and one of the i5's I have came with an upgraded CPU, or splash out and go for the i7.
https://www.aliexpress.com/item/QOTOM-4-LAN-Mini-PC-with-Core-i3-4005U-i5-5250U-processor-and-4-Gigabit-NIC/32812678037.html?spm=2114.search0104.3.1.609f2b45TBugQ0&ws_ab_test=searchweb0_0,searchweb201602_3_10152_10151_10065_10344_10130_10068_10324_10342_10547_10325_10343_5012915_10340_10548_10341_10696_10192_10190_10084_10083_10618_5013015_10307_10820_10301_10821_10303_5012815_10059_100031_5012715_10103_10624_10623_10622_10621_10620,searchweb201603_55,ppcSwitch_3&algo_expid=0456645d-429a-45af-97d4-585e45898742-0&algo_pvid=0456645d-429a-45af-97d4-585e45898742&priceBeautifyAB=0 (https://www.aliexpress.com/item/QOTOM-4-LAN-Mini-PC-with-Core-i3-4005U-i5-5250U-processor-and-4-Gigabit-NIC/32812678037.html?spm=2114.search0104.3.1.609f2b45TBugQ0&ws_ab_test=searchweb0_0,searchweb201602_3_10152_10151_10065_10344_10130_10068_10324_10342_10547_10325_10343_5012915_10340_10548_10341_10696_10192_10190_10084_10083_10618_5013015_10307_10820_10301_10821_10303_5012815_10059_100031_5012715_10103_10624_10623_10622_10621_10620,searchweb201603_55,ppcSwitch_3&algo_expid=0456645d-429a-45af-97d4-585e45898742-0&algo_pvid=0456645d-429a-45af-97d4-585e45898742&priceBeautifyAB=0)
(https://image.ibb.co/mX2JrU/Capture.png)
-
Let me ask something. If someone recommends the APU2 to OP, what reason is behind that recommendation? The OP didnt clarify the bandwidth requirements. It does matter if the APU2 will be serving a 10 Mbit ADSL or a 1Gbit Fiber WAN with PPPoE. It does matter if there will be 2 lines in the firewall ruleset, or 1000. It does matter if IPS/IDS will be activated with many rules, or no IPS/IDS at all.
You are right. That being said, what's the reasoning behind recommending to avoid it without knowing the same? Do you plan to jump in every time someone asks a basic hardware question and recommend avoiding APU devices? Find the device that makes you happy and stick with it.
-
@ricsip
You are right!
But TWB asked explicit for a sbc and for the money the APU2-Boards offers great performance and opportunities (i.e. m-pci slots). So i recommend this board. The iperf may be a problem for you. Others may habe no problem with this!
best regards
Dirk
-
Moved post...
-
Apart from the Qotoms I also own two APU's one's APU1C4 and the other is an APU2C4, both served me well until I got the Qotoms. They are remarkably efficient.
I keep them as reserves, just in case - although it's unlikely I'll ever use the APU1 again.
-
Let me ask something. If someone recommends the APU2 to OP, what reason is behind that recommendation? The OP didnt clarify the bandwidth requirements. It does matter if the APU2 will be serving a 10 Mbit ADSL or a 1Gbit Fiber WAN with PPPoE. It does matter if there will be 2 lines in the firewall ruleset, or 1000. It does matter if IPS/IDS will be activated with many rules, or no IPS/IDS at all.
You are right. That being said, what's the reasoning behind recommending to avoid it without knowing the same? Do you plan to jump in every time someone asks a basic hardware question and recommend avoiding APU devices? Find the device that makes you happy and stick with it.
Beware PCengines APU2, if you want to use it on 1-Gigabit links, where it fails miserably to reach anything close to wire-speed, compared to some Linux-based firewall distrib.
--> Thats what I said. It clearly defines the bottleneck, that the APU2 cannot exceed under opnsense. But it can easily achieve under IPfire.
The positive suggestions by other(s) were based on personal sympathy of the vendor / board itself, not based on the real technical requirements by the OP.
Is that so hard to acknowledge, there is no reliable benchmark on the opnsense wiki, so that one can make sizing based on?
For example, if I were to buy, would you recommend your in-house Deciso DEC600
https://www.deciso.com/product-catalog/dec600/
for the same workload I try to achieve?
Or DEC620? Or DEC2600 ? I really don't know.
-
@ricsip
You are right!
But TWB asked explicit for a sbc and for the money the APU2-Boards offers great performance and opportunities (i.e. m-pci slots). So i recommend this board. The iperf may be a problem for you. Others may habe no problem with this!
best regards
Dirk
"The iperf may be a problem for you"
Its not a problem about iperf. Iperf is simply the tool to confirm there is performance problems with the HW+SW combo under conditions that anybody can possibly face, if signing a contract for 1Gbit internet.
-
The positive suggestions by other(s) were based on personal sympathy of the vendor / board itself, not based on the real technical requirements by the OP.
Hi,
and only you know the exact technical requirements of the OP! Do you!? ;)
OK. An APU2 can't work at 1 Gbit speed with OPNsense!
So let the OP decide if an APU2 meets his requirements or another hw better fits.
best regards
Dirk
PS: Until now I never used or bought ever any hw from pc-engines! I only suggest it because the OP asked for an sbc thats working with OPNsense!
-
The positive suggestions by other(s) were based on personal sympathy of the vendor / board itself, not based on the real technical requirements by the OP.
You think you have present two facts to prove your point, but the first one is an opinion and the second one is a lie by your own admission that the technical requirements were not stated in the first place. You're not making sense and continue your narrative against the best practice of friendly and productive tone in this community.
You're wasting your time and the only thing it gets you is that I'm reminding you of upholding etiquette and suggest to step away from this thread now.
Cheers,
Franco
-
The positive suggestions by other(s) were based on personal sympathy of the vendor / board itself, not based on the real technical requirements by the OP.
You think you have present two facts to prove your point, but the first one is an opinion and the second one is a lie by your own admission that the technical requirements were not stated in the first place. You're not making sense and continue your narrative against the best practice of friendly and productive tone in this community.
You're wasting your time and the only thing it gets you is that I'm reminding you of upholding etiquette and suggest to step away from this thread now.
Cheers,
Franco
If my tone is not friendly (no swearing, not being impolite with anyone) or my warning (it was a warning, nothing more!) is not productive, there are big issues on this forum. Anyway, you achieved your goal against real-life critic: I will not post into this thread anymore.
Btw. the fact you dodged my real question about performance of DECISO products (that have similar specs to PCengines APU2) is also an alarming sign to me.
Have a great day!
-
@ricsip
Even if you can't believe it, you're not the first one noticed that an apu2 does not work for 1 Gbit woth OPNsense. ;)
https://www.calexium.com/en/performance.html
There you'll also find useful information of other hw an OPNsense (also one from DECISO) ;D
best regards
Dirk
-
Btw. the fact you dodged my real question about performance of DECISO products (that have similar specs to PCengines APU2) is also an alarming sign to me.
If you're considering purchasing Deciso products (or even just after some information) then I'd suggest you'd be far better contacting them directly.
-
Many thanks to you all for the great suggestions. I am still unsure of which route I want to take. Regardless, the experience of others in this area is priceless.
I do like the QOTOM rig, I'm not sure I need that much processing power though. Also, I am not totally thrilled with the 15W of power consumption on that box. Not that the "about 6 to 12W depending on CPU load" for the APU2 board is a huge difference in power consumption. Although, anything to save on the power bill each month is a huge bonus for a potential board.
All of that being said, to me, the APU2 boards are looking pretty cool. especially with the red case. 8)
It might actually work for me because I am not even close to gigabit speeds. More like 65Mbps, heh!
I do also like the DECISO Netboard A10. It looks like I can buy one with OPNsense already installed and in a desktop enclosure. However, that is well outside the constraints of my budget. Can I just buy the board and make an enclosure? I didn't find an option to just purchase a single board.
I am still open and interested in other suggestions though (without breaking the bank).
Any thoughts on the MinnowBoard Turbot Dual Ethernet Family?
https://minnowboard.org/minnowboard-turbot-dual-e/
or
The UP Squared board?
https://up-shop.org/28-up-squared
or
Marvell ESPRESSObin? (I know it is not X86_64, still seems somewhat promising)
http://espressobin.net/
Enlightened to new possibilities,
TWB
-
Any thoughts on the MinnowBoard Turbot Dual Ethernet Family?
https://minnowboard.org/minnowboard-turbot-dual-e/
These boards seems to be avaiable only with 2GB RAM. This could be a problem if you want to use IDS/IPS or some plugins with OPNsense.
The UP Squared board?
https://up-shop.org/28-up-squared
Looks nice. But for the same price you can buy IMHO a QOTOM with 4 LAN Ports.
Marvell ESPRESSObin? (I know it is not X86_64, still seems somewhat promising)
http://espressobin.net/
Has no native support for FreeBsd!
https://forum.opnsense.org/index.php?topic=7929.msg36732#msg36732
-
OP is concerned about energy usage, The Qotom when used without the usual energy saving turned on will use approx 16W, that costs about £15 PA at our prices, plus I also have AC Infinity fans on mine, so add a few cents for that too.
As a side note, I did an iPerf test on the Qotom yesterday, it averages around 630Mbps, which ties up with Nivek1612's 1Gb link in France maxing out at around 601Mbps. I also then ran the test using a baremetal VM with Opnsense and then pfSense - just for comparison, could not get anywhere near the 630Mbps with the VM.
-
Thanks again for the responses. I am struggling with a final decision. The Qotom seems like the obvious solution at this point. Regardless of power consumption. It really is a nice box.
I have to ask myself: Can I hold out for freeBSD 12.0 in hopes of getting support for these other boards? I guess only I can honestly answer that question. I feel like I have this dream of creating a cluster of Grapeboards https://www.grapeboard.com/ or maybe some ESPRESSObin 1GB boards http://espressobin.net/ for this project. I can always keep dreaming.
OP is concerned about energy usage, The Qotom when used without the usual energy saving turned on will use approx 16W, that costs about £15 PA at our prices, plus I also have AC Infinity fans on mine, so add a few cents for that too.
Why do you have fans on your rig? Is that something I might have to consider in the future?
Still searching,
TWB
-
It lives in a very enclosed space, very little air movement. Many run them without fans, but with a fan it's stone cold.
-
(Excuses for this very-very long reply)
Even if I clearly saw what happened when @ricsip stated a warning against a specific product based on it's feelings/ emotions, I can't refrain myself from saying Beware and think twice about choosing high performance & high TDP wattage CPU options on allowing POnDesk HW (Products On Desk (Winston Marriot - London UK based) - pondesk.com (http://pondesk.com)) if it might happen to stumble upon it!!!
Only beware! Only mind! Don't avoid at all costs if you really want to try specific high end config! But be prepared for intricacies and stalling and email ping-pongs with sells/ support reps quite a bit. In the end, they did their best to repair the situation, and they did. But with unnecessary statements in emails and trying to dodge the real problem at first.
I can prove anything I state here with emails and pictures, if requested, but I only want to mention the facts for now. So, without further ado, what happened is like following:
I bought a Pondesk NSHO-001 model (https://www.pondesk.com/product/8-LAN-10Gig-Fiber-SFP-4G-NGFW-Firewall-1U-Rackmount-Server_NSHO-001 (https://www.pondesk.com/product/8-LAN-10Gig-Fiber-SFP-4G-NGFW-Firewall-1U-Rackmount-Server_NSHO-001)) with the highest performance CPU available, the I7 - 4790K (4 Cores/ 8 Threads (SMT/ Hyper-Threading), 8 MB cache, 4,4 GHz - 88 W TDP). Actually, I have chosen an I7 - 4790 CPU, but they offered to send the next option as a courtesy of first time customer, and we (me and my corp customer) accept it. (Of course I checked any online reviews I could find about them, but even if I found tens of such reviews, all positive, none were about the 1U rack-mount models, since nobody seemed to have been bought such models from them, but desktop models.)
The reason they offered the I7 K in place of I7 non-K for the same money, I have been finding out later, was that they had one and only one 1U device available, and it was based on that I7 K CPU: the case was dirty (sticky/ oily dirty) and had multiple irregular scratches some of which were quite deep, the rubber pads were already glued on their places excepting one pad that was out of its place, but glue residues was also present on the correct spot nearby... And so on and so forth, suffice to say the product (well over 1000 $) wasn't brand new in its aspect.
OK, it was purchased as a testing unit, so the physical aspect was not an issue, as long as possible following orders will be as they should - but I took pictures of the unit and have sent them back to their sales stating that this physical state for following orders would be a "no go" for us.
First, let's connect a monitor to the device, let's power it up, and enter the BIOS to see whats there: everything OK, reported RAM and storage were as requested/ ordered, the CPU was the K version... Except the reported CPU temperature was a little bit (more) worrying: 55 deg C and rising... Stopped at about 63.
Booted from an utilities USB stick, run some CPU stress testing tools, monitored the temp, and concluded heavy CPU throttling since in just 2-3 seconds since stress test started the reported temp was 100. Repeated tests several times, asked POnDesk if it's OK to check the thermal compound on the CPU cooling unit, they stated ”not necessary, the unit was thoroughly tested, but go ahead, no warranty issues!, opened the unit only to find huge amounts of thermal paste between the CPU and the heatsink, and even more bewildering, none of it was forced out by the pressure, all of it was two thick layers on both the CPU and heatsink, and no metal glimpse could be seen on either of the CPU or heatsink: they didn't mind the mounting position of the heatsink regarding the row of several capacitors near the CPU and one bracket of the heatsink was lying on 2 of those capacitors, one of them having a clear scratch on top, caused by the mounting bracket of the heatsink!!! (!)
Replaced the thermal paste, repositioned the heatsink so that no bracket would lay on any capacitor, only to have marginal improvements, it reached "only" 58 this time during BIOS monitoring, but no difference during stress tests, seconds to reach 100 (only if I would have a car like that ;) ).
Out of pure curiosity, I did install OPNsense on the unit: ~55 on idle, ~85 - 100, mostly 100, on light traffic (only one client, a laptop, with default config, no traffic shaper, no IPS, no NAT... No nothing excepting the default LAN rules). Took some screenshots, sent them on an email, and requested the unit to be repaired/ replaced.
Trying to cut an already long story short, I would say that they accepted the unit to be sent back, but hilariously their comment was that (and mind this) since the OPNsense interface shows 8 cores it is clearly NOT reliable, AND the temp measured is not accurate, because this model of the CPU has only 4 cores!!! Wouldn't you love these salesman people who have not quite all the ideas about what they're selling?!?!? :)))) (Of course, sent them a short response about SMT and Hyper Threading things, directing them to their technical dep, but OH, MY!...)
After a few days they reported the unit just arrived at their address, and that the tech dep will give further details, but for now only resetting the BIOS to defaults lead to peak temp to 55! (Say whaaaat???!!! (!)) I didn't reply on spot. After a few days, I reminded them that no final resolution, nor the further details from tech dep has been sent, and they replied with "We still have temp issues, tests are not finished, and as we don't want to further delay your shipment, we would like to send you our brand new product and design, the NSHO-004 model" (https://www.pondesk.com/product/Intel-Atom-C3758-8-Core-6-LAN-10Gig-SFP-1U-Rackmount-Server_NSHO-004 (https://www.pondesk.com/product/Intel-Atom-C3758-8-Core-6-LAN-10Gig-SFP-1U-Rackmount-Server_NSHO-004)). So not another device of the same config, but an another design by itself - proving insufficient design testing, let alone product testing! (!) My customer agreed, as a second and final chance for this company to prove something, so it arrived after a few more days (this time really being and looking brand new).
All good, temp is as low as possible in between 35 - 45, CPU passive cooling, 2 case fans on the back of the unit, maxing out my customers 1 GB internet fiber link, but no idea about real stressing loads with IPS and/ or VPN since my customer reverted back to virtual appliance and no replacement up to now, only short tests, temp concerned at most, from behind the VM (the actual site-to-site VPN and IPS probable bottleneck).
Both units look like they're presented in the pictures found on website.
Hope it would help someone making the right choice: I would say this NSHO-004 model is quite a good choice for being a Network Appliance powered by OPNsense, at list for what short and superficial tests I ran up to now (I will come back with further details) and since Qotom has no rack-mount units on its offer, I would turn toward NSHO-004 any time if rack-mounting is mandatory.
Also, maybe Deciso would mind the Intel® Atom™ C3758 (8 core, 16M Cache, up to 2.20 GHz) for its products, as those AMD CPUs, even if still notable, are a bit old from now on.
-
Even if I clearly saw what happened when @ricsip stated a warning against a specific product based on it's feelings/ emotions, I can't refrain myself from saying Beware and think twice about choosing high performance & high TDP wattage CPU options on allowing POnDesk HW (Products On Desk (Winston Marriot - London UK based) - pondesk.com (http://pondesk.com)) if it might happen to stumble upon it!!!
Honestly I don't think any of the PONDESK units will work for me. The only one that looks remotely close to what I am after is the PONDESK PICO PC (https://www.pondesk.com/mini-server-and-router/). Unfortunately, they are a far cry from inside my current budget.
Thank you for the suggestion though,
TWB
-
Hello again everyone,
I just found the fitlet2 from Compulab and I am really liking this box. Any thoughts?
https://fit-iot.com/web/products/fitlet2/fitlet2-specifications/
I know it only has the 2 Ethernet ports and that's alright with me. Whether I have 4, 3, or 2 everything goes to through a switch anyway.
There is also the SBC-FLT, which I am not totally sure I like that much.
http://www.fit-pc.com/web/products/fitlet/sbc-flt/
On a side note, while you are at the fit-iot website, you should check out the Airtop2 if you haven't seen it before. Built to order box of awesome!
https://fit-iot.com/web/products/airtop2/airtop2-specifications/
Still searching,
TWB
-
Looks good, but there's only one way to find out.
Oh BTW, I was quite happy with two NICs, until I needed three! :)
-
Oh BTW, I was quite happy with two NICs, until I needed three! :)
If your HW offers a free mPCEe-slot you can use a mPCIe2LAN Adapter. ;)
i.e.: https://www.amazon.de/Informatique-CONTROLEUR-Express-MiniPCIe-Ethernet/dp/B01NBRA4IX/ref=sr_1_3?ie=UTF8&qid=1534744172&sr=8-3&keywords=mini+pcie+lan
@TWB
It's not easy to find the best hw solution. Everyone needs other conditions. ;)
For me, my hw has to offer the following things: cheap, small and low power consumption. ;) At least enough performance for my adsl connection (100/20 Mbit).
Since my first steps to OPNsense i use old hw from commercial fw solutions vendors (i.e. Securepoint, Astaro/Sophos, Gateprotect, Ucopia). These hw often sold by ebay (europe) for a handfull of €.
Right now i use an Ucopia Express 250 Mainboard into a Securepoint Black Dwarf G1 case. Both fw build by the same oem (Lexcom). So the Mainboard fits into case without any problems.
Power Consumtion is around 10W.
best regards
Dirk