Fortunately, I have a HomeLab specifically for testing these sorts of things first. It's also entirely possible I misread or skimmed over something in one (or more) of the upgrade pop-ups that could have made this go a lot smoother.
Relevant things to know ahead of time:
- I'm a native Ubuntu user for nearly 20 years. I'm comfortable in a terminal, but not BSD-native.
- My HomeLab OPNsense install is router-on-a-stick layout; there's one physical NIC but everything is VLANs.
- The NIC hardware is some form of Realtek; I had the os-realtek-re package installed previously (not sure if it disabled on this upgrade or an older one).
- I use DDNS and WireGuard to VPN into this network.
- I have several DHCP reservations for the LAN-side VLAN.
- I have multi-WAN setup.
- There are two physical machines - the one I tried to upgrade, that's still broke, I'll call it "old_hardware", and a machine I pulled out of storage to at least try and get back up and running a bit ("new_hardware").
The firewall old_hardware appeared to just never come back up from the upgrade. No DHCP, no establishing the WireGuard tunnel, nothing. Plugging in a keyboard/monitor indicated that the interface wasn't coming up properly. When I reboot it, I can see it saying "re0 up" in the boot logs, but it doesn't actually come up (doesn't list WAN VLANs as having received DHCP IPs, they're just blank). In fact, it's so "down", that the switch it's plugged into, never even lights up the lights for that port. The new_hardware firewall plugged into this same port, with the same cable, works fine. I've never had an issue with the physical NIC on old_hardware until now - only driver/software issues, so I don't think that's what's happening this time, either.
I downloaded the amd64 DVD ISO and tried to put it on a flash drive to install on new_hardware, but neither dd- nor cat-created UFDs were able to boot; I had to drop the ISO into a Ventoy drive to be able to boot it at all. (26.1.6). (Yes, I validated the checksum and then unzipped it first.)
At one point I tried a USB Ethernet adapter (ASIX chipset) in the old_hardware machine, but it didn't seem to have the kernel module needed for it.
Getting the config.xml onto the new hardware for a Linux-native seems... much more difficult than it should be. I tried booting a Live Ubuntu instance (on the freshly-OPNsense-installed new_hardware machine) and importing the zroot, but it was just a handful of empty folders. I tried formatting a USB Flash Drive (UFD) with Ubuntu as MBR/vfat and the "b" partition type, but that didn't seem to work (couldn't get it to mount, neither with "mount" nor "mount_msdosfs"). Eventually, I wiped/formatted/filesystem'd the UFD using the OPNsense instance on new_hardware, and then sneakernet'd that over to an Ubuntu box, which was able to read, and place, the config.xml I needed; then sneakernet back, mount, and I had my config imported (after a reboot).
Upon booting this imported config on new_hardware, I still had several issues, such as:
- no DHCP working at all. The ISC package was removed, and Kea was not enabled.
- no DDNS. The package was removed.
- os-realtek-re not available at all; I still can't find it in either Packages or Plugins, even after another update/upgrade and re-check (26.1.7 now).
- no export/migration option for ISC-->Kea. Allegedly there's supposed to be a CSV export/import function, but I can't find it.
I'm half tempted to just pull the drive from new_hardware and drop it into old_hardware and see what happens. I'm also tempted to install a couple major versions older than current, import the config, migrate all the DHCP stuff over, and retry the upgrade.
I'm not sure the best course of action, but I've already spent 4 hours of troubleshooting and another 45 minutes of documenting of said troubleshooting tonight, so I'm looking for the "easiest" path forward to have a functional old_hardware system again. Note that transplanting the HDD from new_hardware now will NOT give me that, due to being stuck on ISC with no way to export/import to Kea, even though I did fix the DDNS issue.
Also going forward - is there a way to force packages to auto-reinstall after system upgrades, especially critical things like drivers or DDNS?
When restoring an older config with plugins the plugins need to be added from the GUI first. That's a problem now for ISC-DHCP in use indeed, but given it's deprecation state we don't have another choice if we don't want to drag it around for a few more years in the default install (which upon final removal would leave people in the same state as you describe still).
The rest are more or less recurring setup issues people can run into, but nothing concrete.
Start from the factory image first and make sure to install all required plugins beforehand. Then import, don't let it reboot just yet and do a final settings check, then reboot. If the settings aren't perfect a console helps to recover or make the necessary adjustments.
Cheers,
Franco