TL;DR:
I installed OPNsense on a Dell R620 which should be massive overkill, but it's painfully slow and having problems that normally just don't happen. Are there known issues with OPNsense on Dells, or anything that needs to be done starting fro ma fresh install?
Note that this is bare metal, no virtualization.
Background:
I'm trying to find a good host for an OPNsense router noting that I have a bajillion VLANs and 25 Gbe internal networking.
Previously I tried a miniforums MS-01 and that was AWFUL. So I figured for sure a Dell server would be a quick solution that would just work while I tried other more questionable solutions.
Unfortunately it's anything but! I was able to do the installation to a USB (as I normally do) which was slow, but that's a one time thing. However running was painfully slow:
But for instance if I'm at the command interface and I press '8' to open a shell, it took 21.86 seconds to load a shell and present a prompt.
The process of setting the IP of a network interface was 1 minute 33 seconds -- 37.09 seconds of which were walking through the prompts of me not changing anything (for those who want to accuse me of being a slow typist).
I also never even GOT to the web interface because I couldn't get it started. One problem I did notice was that when I changed the network interface & set an IP, the route to the IP stayed on the old physical interface -- for instance by default igb0 was WAN & igb1 was LAN but I wanted to change LAN to igb2. When I made that change the route 192.168.1.0/24 stayed on igb1 even though nothing (LAN nor WAN) was on igb1 so naturally the 'interface was down'. Changing the interface manually seems to have worked enough for pinging to work but not enough to trigger the webgui to start.
I'm retrying now on a HDD instead of USB (sadly these) as I did find a post saying USB drivers can be slow. But that's a little confusing to me as I thought OPNSense lived in RAM while running.
Any recommendations on how to get this hardware to work or reasons this hardware won't work (if there's an issue, I'll happily take recommendations)? See specs below.
Specs:
Dell R620
Dual Intel(R) Xeon(R) CPU E5-2650L v2 @ 1.70GHz
192 GB DDR3 ECC RAM
Hmm
It should run beautifully on it.
Yes if you're going to run bare metal, DO install the OPNsense OS to a disk or disk if you have a PERC card.
I recommend using EFI instead of legacy BIOS/MBR booting.
The rest should load. For some reason whether decided by FreeBSD or Deciso devs, not all network cards will PnP and automatically load their corresponding driver.
You may need to add tunable that tells the underlying OS to load its driver for hardware like your 25Gb cards.
Definitely don't run off USB.
Ok, tried installing to disk and it's working beautifully now. I'm sure going to HDD over USB helped, but it feels like it was just a bad install.
I do indeed have a PERC card, but still only installed to one disk to avoid config issues (I really just want to get this working at the moment, happy to experiment later). I did notice that ZFS seems to be the default now, I had a terrible experience with that a few years ago where USB drives did NOT like it (I was only trying to do a Zmirror) and kept getting "corrupted" (they were recoverable by reformatting) but going back to UFS fixed it instantly. I take it ZFS is seen as stable now? Needless to say this is not a ZFS install at the moment.
The good news is that OPNsense does show me the interfaces (mce0 & mce1) on the card. The bad news is that even though they report to be up in the assignments page (green icon instead of red), the VLANs on top all seem to be down. I think there's a trick I'm missing, but that's a tomorrow me problem!
I've got most of my config moved over, so I'm calling this a win for the night.