Question About Hardware Compatibility - UPDATE 8/14/2018

[marjohn56]

Apart from the Qotoms I also own two APU's one's APU1C4 and the other is an APU2C4, both served me well until I got the Qotoms. They are remarkably efficient.


I keep them as reserves, just in case - although it's unlikely I'll ever use the APU1 again.

[Ricardo]

Let me ask something. If someone recommends the APU2 to OP, what reason is behind that recommendation? The OP didnt clarify the bandwidth requirements. It does matter if the APU2 will be serving a 10 Mbit ADSL or a 1Gbit Fiber WAN with PPPoE. It does matter if there will be 2 lines in the firewall ruleset, or 1000. It does matter if IPS/IDS will be activated with many rules, or no IPS/IDS at all.

You are right. That being said, what's the reasoning behind recommending to avoid it without knowing the same? Do you plan to jump in every time someone asks a basic hardware question and recommend avoiding APU devices? Find the device that makes you happy and stick with it.

Beware PCengines APU2, if you want to use it on 1-Gigabit links, where it fails miserably to reach anything close to wire-speed, compared to some Linux-based firewall distrib.
--> Thats what I said. It clearly defines the bottleneck, that the APU2 cannot exceed under opnsense. But it can easily achieve under IPfire.

The positive suggestions by other(s) were based on personal sympathy of the vendor / board itself, not based on the real technical requirements by the OP.

Is that so hard to acknowledge, there is no reliable benchmark on the opnsense wiki, so that one can make sizing based on?
For example, if I were to buy, would you recommend your in-house Deciso DEC600
https://www.deciso.com/product-catalog/dec600/
for the same workload I try to achieve?
Or DEC620? Or DEC2600 ? I really don't know.

[Ricardo]

@ricsip
You are right!
But TWB asked explicit for a sbc and for the money the APU2-Boards offers great performance and opportunities (i.e. m-pci slots). So i recommend this board. The iperf may be a problem for you. Others may habe no problem with this!

best regards
Dirk

"The iperf may be a problem for you"
Its not a problem about iperf. Iperf is simply the tool to confirm there is performance problems with the HW+SW combo under conditions that anybody can possibly face, if signing a contract for 1Gbit internet.

[monstermania]

The positive suggestions by other(s) were based on personal sympathy of the vendor / board itself, not based on the real technical requirements by the OP.

Hi,
and only you know the exact technical requirements of the OP! Do you!? 
OK. An APU2 can't work at 1 Gbit speed with OPNsense!
So let the OP decide if an APU2 meets his requirements or another hw better fits.

best regards
Dirk

PS: Until now I never used or bought ever any hw from pc-engines! I only suggest it because the OP asked for an sbc thats working with OPNsense!

[franco]

The positive suggestions by other(s) were based on personal sympathy of the vendor / board itself, not based on the real technical requirements by the OP.

You think you have present two facts to prove your point, but the first one is an opinion and the second one is a lie by your own admission that the technical requirements were not stated in the first place. You're not making sense and continue your narrative against the best practice of friendly and productive tone in this community.

You're wasting your time and the only thing it gets you is that I'm reminding you of upholding etiquette and suggest to step away from this thread now.


Cheers,
Franco

[Ricardo]

The positive suggestions by other(s) were based on personal sympathy of the vendor / board itself, not based on the real technical requirements by the OP.

You think you have present two facts to prove your point, but the first one is an opinion and the second one is a lie by your own admission that the technical requirements were not stated in the first place. You're not making sense and continue your narrative against the best practice of friendly and productive tone in this community.

You're wasting your time and the only thing it gets you is that I'm reminding you of upholding etiquette and suggest to step away from this thread now.


Cheers,
Franco

If my tone is not friendly (no swearing, not being impolite with anyone)  or my warning (it was a warning, nothing more!) is not productive, there are big issues on this forum. Anyway, you achieved your goal against real-life critic: I will not post into this thread anymore.

Btw. the fact you dodged my real question about performance of DECISO products (that have similar specs to PCengines APU2) is also an alarming sign to me.

Have a great day!

[monstermania]

@ricsip
Even if you can't believe it, you're not the first one noticed that an apu2 does not work for 1 Gbit woth OPNsense. 
https://www.calexium.com/en/performance.html
There you'll also find useful information of other hw an OPNsense (also one from DECISO) 

best regards
Dirk

[phoenix]

Btw. the fact you dodged my real question about performance of DECISO products (that have similar specs to PCengines APU2) is also an alarming sign to me.

If you're considering purchasing Deciso products (or even just after some information) then I'd suggest  you'd be far better contacting them directly.

[TheWhiteBuffalo]

Many thanks to you all for the great suggestions. I am still unsure of which route I want to take. Regardless, the experience of others in this area is priceless.

I do like the QOTOM rig, I'm not sure I need that much processing power though. Also, I am not totally thrilled with the 15W of power consumption on that box. Not that the "about 6 to 12W depending on CPU load" for the APU2 board is a huge difference in power consumption. Although, anything to save on the power bill each month is a huge bonus for a potential board.

All of that being said, to me, the APU2 boards are looking pretty cool. especially with the red case. 
It might actually work for me because I am not even close to gigabit speeds. More like 65Mbps, heh!

I do also like the DECISO Netboard A10. It looks like I can buy one with OPNsense already installed and in a desktop enclosure. However, that is well outside the constraints of my budget. Can I just buy the board and make an enclosure? I didn't find an option to just purchase a single board.

I am still open and interested in other suggestions though (without breaking the bank).

Any thoughts on the MinnowBoard Turbot Dual Ethernet Family?
https://minnowboard.org/minnowboard-turbot-dual-e/

or

The UP Squared board?
https://up-shop.org/28-up-squared

or

Marvell ESPRESSObin? (I know it is not X86_64, still seems somewhat promising)
http://espressobin.net/

Enlightened to new possibilities,

TWB

[monstermania]

Any thoughts on the MinnowBoard Turbot Dual Ethernet Family?
https://minnowboard.org/minnowboard-turbot-dual-e/

These boards seems to be avaiable only with 2GB RAM. This could be a problem if you want to use IDS/IPS or some plugins with OPNsense.

The UP Squared board?
https://up-shop.org/28-up-squared

Looks nice. But for the same price you can buy IMHO a QOTOM with 4 LAN Ports.

Marvell ESPRESSObin? (I know it is not X86_64, still seems somewhat promising)
http://espressobin.net/

Has no native support for FreeBsd!
https://forum.opnsense.org/index.php?topic=7929.msg36732#msg36732

[marjohn56]

OP is concerned about energy usage, The Qotom when used without the usual energy saving turned on will use approx 16W, that costs about £15 PA at our prices, plus I also have AC Infinity fans on mine, so add a few cents for that too.


As a side note, I did an iPerf test on the Qotom yesterday, it averages around 630Mbps, which ties up with Nivek1612's 1Gb link in France maxing out at around 601Mbps. I also then ran the test using a baremetal VM with Opnsense and then pfSense - just for comparison,  could not get anywhere near the 630Mbps with the VM.

[TheWhiteBuffalo]

Thanks again for the responses. I am struggling with a final decision. The Qotom seems like the obvious solution at this point. Regardless of power consumption. It really is a nice box.

I have to ask myself: Can I hold out for freeBSD 12.0 in hopes of getting support for these other boards? I guess only I can honestly answer that question. I feel like I have this dream of creating a cluster of Grapeboards https://www.grapeboard.com/ or maybe some ESPRESSObin 1GB boards http://espressobin.net/ for this project. I can always keep dreaming.

OP is concerned about energy usage, The Qotom when used without the usual energy saving turned on will use approx 16W, that costs about £15 PA at our prices, plus I also have AC Infinity fans on mine, so add a few cents for that too.

Why do you have fans on your rig? Is that something I might have to consider in the future?

Still searching,

TWB

[marjohn56]

It lives in a very enclosed space, very little air movement. Many run them without fans, but with a fan it's stone cold.

[Ciprian]

(Excuses for this very-very long reply)

Even if I clearly saw what happened when @ricsip stated a warning against a specific product based on it's feelings/ emotions, I can't refrain myself from saying Beware and think twice about choosing high performance & high TDP wattage CPU options on allowing POnDesk HW (Products On Desk (Winston Marriot - London UK based) - pondesk.com) if it might happen to stumble upon it!!!

Only beware! Only mind! Don't avoid at all costs if you really want to try specific high end config! But be prepared for intricacies and stalling and email ping-pongs with sells/ support reps quite a bit. In the end, they did their best to repair the situation, and they did. But with unnecessary statements in emails and trying to dodge the real problem at first.

I can prove anything I state here with emails and pictures, if requested, but I only want to mention the facts for now. So, without further ado, what happened is like following:

I bought a Pondesk NSHO-001 model (https://www.pondesk.com/product/8-LAN-10Gig-Fiber-SFP-4G-NGFW-Firewall-1U-Rackmount-Server_NSHO-001) with the highest performance CPU available, the I7 - 4790K (4 Cores/ 8 Threads (SMT/ Hyper-Threading), 8 MB cache, 4,4 GHz - 88 W TDP). Actually, I have chosen an I7 - 4790 CPU, but they offered to send the next option as a courtesy of first time customer, and we (me and my corp customer) accept it. (Of course I checked any online reviews I could find about them, but even if I found tens of such reviews, all positive, none were about the 1U rack-mount models, since nobody seemed to have been bought such models from them, but desktop models.)

The reason they offered the I7 K in place of I7 non-K for the same money, I have been finding out later, was that they had one and only one 1U device available, and it was based on that I7 K CPU: the case was dirty (sticky/ oily dirty) and had multiple irregular scratches some of which were quite deep, the rubber pads were already glued on their places excepting one pad that was out of its place, but glue residues was also present on the correct spot nearby... And so on and so forth, suffice to say the product (well over 1000 $) wasn't brand new in its aspect.

OK, it was purchased as a testing unit, so the physical aspect was not an issue, as long as possible following orders will be as they should - but I took pictures of the unit and have sent them back to their sales stating that this physical state for following orders would be a "no go" for us.

First, let's connect a monitor to the device, let's power it up, and enter the BIOS to see whats there: everything OK, reported RAM and storage were as requested/ ordered, the CPU was the K version... Except the reported CPU temperature was a little bit (more) worrying: 55 deg C and rising... Stopped at about 63.

Booted from an utilities USB stick, run some CPU stress testing tools, monitored the temp, and concluded heavy CPU throttling since in just 2-3 seconds since stress test started the reported temp was 100. Repeated tests several times, asked POnDesk if it's OK to check the thermal compound on the CPU cooling unit, they stated ”not necessary, the unit was thoroughly tested, but go ahead, no warranty issues!, opened the unit only to find huge amounts of thermal paste between the CPU and the heatsink, and even more bewildering, none of it was forced out by the pressure, all of it was two thick layers on both the CPU and heatsink, and no metal glimpse could be seen on either of the CPU or heatsink: they didn't mind the mounting position of the heatsink regarding the row of several capacitors near the CPU and one bracket of the heatsink was lying on 2 of those capacitors, one of them having a clear scratch on top, caused by the mounting bracket of the heatsink!!! (!)

Replaced the thermal paste, repositioned the heatsink so that no bracket would lay on any capacitor, only to have marginal improvements, it reached "only" 58 this time during BIOS monitoring, but no difference during stress tests, seconds to reach 100 (only if I would have a car like that ).

Out of pure curiosity, I did install OPNsense on the unit: ~55 on idle, ~85 - 100, mostly 100, on light traffic (only one client, a laptop, with default config, no traffic shaper, no IPS, no NAT... No nothing excepting the default LAN rules). Took some screenshots, sent them on an email, and requested the unit to be repaired/ replaced.

Trying to cut an already long story short, I would say that they accepted the unit to be sent back, but hilariously their comment was that (and mind this) since the OPNsense interface shows 8 cores it is clearly NOT reliable, AND the temp measured is not accurate, because this model of the CPU has only 4 cores!!! Wouldn't you love these salesman people who have not quite all the ideas about what they're selling?!?!? )) (Of course, sent them a short response about SMT and Hyper Threading things, directing them to their technical dep, but OH, MY!...)

After a few days they reported the unit just arrived at their address, and that the tech dep will give further details, but for now only resetting the BIOS to defaults lead to peak temp to 55! (Say whaaaat???!!! (!)) I didn't reply on spot. After a few days, I reminded them that no final resolution, nor the further details from tech dep has been sent, and they replied with "We still have temp issues, tests are not finished, and as we don't want to further delay your shipment, we would like to send you our brand new product and design, the NSHO-004 model" (https://www.pondesk.com/product/Intel-Atom-C3758-8-Core-6-LAN-10Gig-SFP-1U-Rackmount-Server_NSHO-004). So not another device of the same config, but an another design by itself - proving insufficient design testing, let alone product testing! (!) My customer agreed, as a second and final chance for this company to prove something, so it arrived after a few more days (this time really being and looking brand new).

All good, temp is as low as possible in between 35 - 45, CPU passive cooling, 2 case fans on the back of the unit, maxing out my customers 1 GB internet fiber link, but no idea about real stressing loads with IPS and/ or VPN since my customer reverted back to virtual appliance and no replacement up to now, only short tests, temp concerned at most, from behind the VM (the actual site-to-site VPN and IPS probable bottleneck).

Both units look like they're presented in the pictures found on website.

Hope it would help someone making the right choice: I would say this NSHO-004 model is quite a good choice for being a Network Appliance powered by OPNsense, at list for what short and superficial tests I ran up to now (I will come back with further details) and since Qotom has no rack-mount units on its offer, I would turn toward NSHO-004 any time if rack-mounting is mandatory.

Also, maybe Deciso would mind the Intel® Atom™ C3758 (8 core, 16M Cache, up to 2.20 GHz) for its products, as those AMD CPUs, even if still notable, are a bit old from now on.

[TheWhiteBuffalo]

Even if I clearly saw what happened when @ricsip stated a warning against a specific product based on it's feelings/ emotions, I can't refrain myself from saying Beware and think twice about choosing high performance & high TDP wattage CPU options on allowing POnDesk HW (Products On Desk (Winston Marriot - London UK based) - pondesk.com) if it might happen to stumble upon it!!!

Honestly I don't think any of the PONDESK units will work for me. The only one that looks remotely close to what I am after is the PONDESK PICO PC. Unfortunately, they are a far cry from inside my current budget.

Thank you for the suggestion though,

TWB