Messages

Is noise a concern?

I'm not space constrained.  As I mentioned, I'm using an old desktop currently.  However, the desktop requires add in cards.  The cards and case fans add to the power budget.  I do need to figure out how much power each of those draw.  I seem to recall that the newer versions of my NICs use less power.

I'm contemplating building an i3-12100 system but I don't know if it's worth the hassle.  The only real issue with the power right now is runtime on my UPS and the power doesn't go out that often.

Does (for example) an Intel 10Gb chip and two SFP+ ports automatically pull more power if they are on a PCIe card vs soldered straight to the motherboard? A single 120mm fan typically pulls 1-2W (at speed). I think choice of CPU and NIC are going to be your largest determining factors in power draw. Something fast enough to cater for your 10Gb needs but as power efficient as possible. That points to latest gen, and/or embedded/laptop CPUs if you can find them on a suitable motherboard or firewall appliance.

AMDs next generation of embedded Epyc should be interesting when they arrive. Current ones are Zen2 and getting quite old. But SuperMicro has them in interesting motherboard combinations, compact form factor and with 10Gb networking.

Yep, but keep in mind the sole purpose of V1500B was to offer cheaper access to 10G. It's not about throughput... it's about connectivity.

Yes agreed, and it is very small and relatively cheap and also fanless, so definitely fills an important gap. Just saying if you're willing to forego one of those (in this case size and to lesser extent noise) there are alternatives which offer more power at similar price points. As one of the OP's concerns was performance.

PS I believe the DEC750 uses Ryzen Embedded V1500B. Which in comparison is old (2018) and underpowered. It's got a passmark of approx 1200/5200 single/multithreaded. Current gen Ryzen 5600x measures approx 3300/22000 by comparison and still idles approx 20W.

Are you space constrained? If not you could assemble a system yourself, using tower or rack case of choice, and if you combine with decent fans (e.g. Noctua or BeQuiet) you can make it whisper quiet too. Motherboards could be for example Supermicro X12STL-F (for Intel Xeon E-2300) or ASRock Rack X570D4U (for latest gen AMD Ryzen CPUs). These are modern CPUs that should sip no more than 30ish watts at idle, if that. Throw in an Intel or Chelsio 10Gb PCIe card and you've got a fast, power efficient and quiet setup.

PS re SFP+ compatibility, I think it's more a question of driver/hardware than support in OpnSense. So you may want to look closer at the specs and recommendations for those cards mentioned. FWIW, I have used fs.com transceivers with great success. Alas have never attempted 2.5 or 5Gb - only 10Gb or 1Gb, and mostly with Chelsio cards. I understand Intel cards are a bit more picky.

From the various performance discussions on these forums relating to 10Gb, it seems native netmap support is recommended for performance, and that currently limits your options more or less to Intel or Chelsio.

See https://www.freebsd.org/cgi/man.cgi?query=netmap&sektion=4

Are you buying new or used? Look for Chelsio T520 or Intel X520 or X710.

I guess I can add for completeness. Upgraded today, performance unchanged as far as I can measure it. Forwarding throughput still tops out just south of 5Gb/s for me.

Thanks, that looks a bit promising, some 15% gain or so.

And yes, agree on the test automation. That would also catch accidental regression. I understand that catching all cases and edge cases would be complex, but some relatively basic throughout testing on a few sets of typical hardware sounds like something a devops whiz could pull together relatively easily? ;-)

Hi all,
I'm itching to upgrade to 22.1 particularly for FreeBSD 13 and the potential performance improvements that come with it. Curious - has anyone measured and done this comparison in OpnSense?

My setup is 10Gb WAN and mix 10Gb/1Gb LAN, Chelsio T520 dual 10Gb and Intel I350-T4 passthrough in Proxmox VM, Ryzen 3700x 8 cores on AMD X470, not much else running on the same box. My throughput currently tops out around 4-5Gb/s but I should be able to get more.

Dear moderators. Please close this topic. It's getting out of hand. I specifically mentioned that this is not a ZFS usage question. Not sure why people can't read a question properly. This means I do not need any comments on how safe it is to use ZFS w/o ECC.

With respect. Your question was

how do people use ZFS with OPNsense

... in the context of how hard it is to find ECC hardware. The problem is that this basic assertion and connection that you are making (ZFS without ECC ram = bad) is false and therefore the whole premise of your question is problematic. The responses you got were towards that end and they were factual and helpful, but then you switch topic and start talk about (lack of) recovery tools instead. Which is a completely different point that has nothing to do with either ecc ram, firewall hardware or OpnSense.

And then...

No matter what someone says, using ZFS without ECC is a bad idea if you care about your data. Nobody will ever convince me otherwise. Period. (I also mentioned that ZFS is not less safe than any other fs. I said that it was much more likely that your data is gone and non-recoverable (in case of a corruption), because of the lack of proper recovery tools for ZFS.)

... you're back at this again. And I still don't understand what your question is, if it isn't about ECC ram. And, by the way, you're wrong ;)

Before you get too frustrated with the answers you're getting, maybe try again and express a question more clearly and maybe you'll get answers that you find more helpful.

I am not saying that another fs is safer because it is less prone to corruption. I am very well aware that e.g. ext4 can be as easily be corrupted as ZFS in case of mem errors.
The issue lies somewhere else. ZFS does not have proper recovery tools. When you have a filesystem corruption with ZFS (especially when the entire pool is affected), it's much more likely that your data is gone and non-recoverable. But I seriously don't want to discuss filesystem coding here.

But what has any of that to do with ECC, which was your question (how do people run ZFS without ECC)...? Are you asking more generally if ZFS is more or less safe than other filesystems, as function of recovery tools (or not)?

Irrespective of recovery tools and the usefulness thereof, remember that ZFS has several built-in mechanisms to avoid you getting there in the first place, including checksums and redundant copies of metadata (and data if you like), scrubbing, copy-on-write at both disk management and filesystem level, etc. In any case, as for the general merits and weaknesses of ZFS (recovery tools or not), the Internet is awash with resources and needless to say ZFS has been thoroughly battle tested in very large and demanding environments for a very long time (with its roots in Solaris some 20 years ago).

However, I just had another thought. I should be fine with a corrupted ZFS on a FW appliance. Worst case scenario: I re-install and restore a config backup.

Well, yes. But again, I don't know why you worry about ZFS corruption. It should be the better choice if anything. Far more likely that some other part of your hardware breaks or that you botch up your system with a failed upgrade or whatever.

As it happens I have ECC ram but would still use ZFS even if I didn't.

The "ZFS without ECC will destroy your data" idea was pushed very strongly on the Free/TrueNAS forums but has been pretty throughly debunked by the ZFS dev community by now. Even the TrueNAS folks seems to have accepted this. I wouldn't worry too much. ZFS is a fine choice. Your future problems are much more likely to come from somewhere else.

Cool! You have many hours of tinkering ahead. I'm in a similar position myself although slightly ahead of you. It's very rewarding as your setup grows and you can expand into new areas and discover useful applications to run and host for yourself and friends and family.

My advise for what it's worth would be to avoid putting all eggs in one basket. Keep your edge router and firewall on separate hardware from your other services so you don't lose all internet and internal routing when you update your main hypervisor. For that purpose the D-1518 would do just fine, or you could get away with something smaller too (Qotom are popular here). Run OpnSense bare metal, or virtualized on ESXi or Proxmox - if virtualized you build more complexity but gain some benefits in easy backup and rollback etc.

Then install Proxmox or ESXi on additional hardware and add VMs and services to taste including TrueNAS. When you run out of hardware, add another node and create a cluster - and then you're already some way into the "homelab" rabbit hole before you even realized ;-)

YMMV of course.

OpnSense on Xeon should be very well supported overall. As for the D-1518, it's quite old (6 years?) and quite power efficient but not particularly fast compared to current gen CPUs. Passmark gives it score of 1256 per core and 4784 total. In direct comparison AMD Epyc 3201 has the same power budget (~30W) and 1928 per core and 10258 total. Desktop/server CPUs will be faster still, but more power hungry.

All that said, the 1518 should still have no problem driving 1Gb/s including IPS/IDS, but it may not scale so well if you want to add additional stuff on top.

As for SFP(+), you can easily add such ports by adding a NIC in a PCIe slot. You can get server level stuff relatively cheap 2nd hand. So don't stare yourself blind on having it built-in.

Also, it may not be so easy cutting out the ISP's modem. ISPs sometimes use GPON or equivalent which means your modem does some heavy lifting to filter out the traffic that relates only to your particular connection, and this is not easy to replicate in your own setup (by design). I'd recommend you do some research on that first before you purchase your own hardware, possibly only to find that you can't get a link on the fiber connection.