Another 26.7 "Upgrade Failed" Thread.

[Kevin G.]

New to the forums, I did check many other previous posts. Have been searching for over a week. Attempted the upgrade last weekend, trying to revisit this weekend...

From My Reddit Post:

Baremetal, i5/3rd Gen, 16GB Ram. ZFS

I have been sweating the update for weeks. Saw several posts and prepared myself the best I could. Attempted the upgrade tonight and lost all connectivity. Build: ZFS (snapshot before upgrade), never had microcode plugin installed, Legacy ISC, simple rules migrated over, backed up Config. Began the upgrade.

Got stuck in a "rebooting now" and returning to the Lobby/Dashboard loop, assumed the update failed. Started to disconnect to add my ISP/backup router back in to troubleshoot, decided to manually reboot the Opnsense box, patiently waited, and heard the boot chime. Logged in from my daily driver desktop, and saw the new dashboard with 26.7.1. Loaded rules via the migration tool, verified they were there, deleted "Legacy Rules" via the tool. Success! ...Nope, no devices on my network receiving any connection. I cannot find the issue, but the Live logs show no traffic being let in whatsoever (all blocked).

I have no clue how to install the bootloader to the secondary drive. Reading tons on it, but just don't understand any of it.

Switched to my backup/ISP provided router, for now...hope to dive in some more tomorrow.

Any insights would be great.

Feels like this is simply a rules issue, but don't know where to begin.

If I need a fresh install, that isn't a big deal, I just have a few questions.
Will I still be able to migrate my old rules?
Are there "new" default rules with a fresh install?
Will my old 26.1 config files work with a fresh install of 26.7 (Legacy ISC and Legacy Rules)? I hope to migrate to dnsmasq, but would prefer to be running first.

[cookiemonster]

with loads of successful upgrades so far I can only assume some sort of setup/settings have come to bite.
What makes you think that you need to install the bootloader to the secondary drive? Do you have a mirrored pair of disks for the OS ? If yes, then it is a likely possibility but if you can boot after the upgrade, then you can look at that afterwards.
What's the status now, is it rolledback via a snapshot?

Note that if you had to put the isp router back in and then had router-behind-a-router, then that could account for no clients hitting it.

[Kevin G.]

with loads of successful upgrades so far I can only assume some sort of setup/settings have come to bite.

Thank you for the reply.

What makes you think that you need to install the bootloader to the secondary drive?

It was just part of what I have been reading needs to be done to get ZFS corrected?

Do you have a mirrored pair of disks for the OS ? If yes, then it is a likely possibility but if you can boot after the upgrade, then you can look at that afterwards.

Yes, and i can boot to OpnSense and look at (what appears to be) a working build...other than no internet access.

What's the status now, is it rolledback via a snapshot?

No, it's completely in the state of the upgrade and migrated rules. UI works, I get the new dashboard, Rules appear to be in place...I had too many questions to give up on it, or attempt a fresh install. Someone on the app thought it might be the kernel fail...but it doesn't (didn't) stay in the boot loop. When I manually restarted, the upgrade must have continued (or I hadn't waited long enough).

Note that if you had to put the isp router back in and then had router-behind-a-router, then that could account for no clients hitting it.

I only put the ISP provided router in to get things working. My wife works from home, so I need a contingency to keep her up and running. The OpnSense box is offline while the ISP router is in place. I can only devote time to the fix late at night, or on the weekends.

[Patrick M. Hausen]

What makes you think that you need to install the bootloader to the secondary drive?

How are you going to boot from the secondary drive when the primary one fails?

[cookiemonster]

What makes you think that you need to install the bootloader to the secondary drive?

How are you going to boot from the secondary drive when the primary one fails?

He mentioned a secondary drive so I assumed a mirror.

[Patrick M. Hausen]

Yes, of course. And with a mirror you need a boot loader on both drives. Which the FreeBSD installer up to a couple of days ago installed on the first drive only. So you need to install it on the second one afterwards, manually.

Also in case of a boot loader update you need to do that on both drives.

The boot loader partitions are not mirrored!

[bamf]

He mentioned a secondary drive so I assumed a mirror.

Just because it's a ZFS mirror doesn't mean both drives are bootable. The installer leaves the EFI partition on the second drive unformatted, so if you don't create the filesystem and copy the bootloader over manually, it won't boot when the primary drive fails. Learned that here.

[cookiemonster]

I don't know why I'm being quoted since we're agreeing and saying the same thing. The advice is for the OP.

[Patrick M. Hausen]

Originally you told the OP:

What makes you think that you need to install the bootloader to the secondary drive?

And the answer is that a ZFS mirror does not mirror the boot loader. So we are trying to correct that downright dangerous advice of yours.
If you run a ZFS mirror you absolutely must install the boot loader on the second drive because the OPNsense installer does not do that.

Clear now?

[drosophila]

Got stuck in a "rebooting now" and returning to the Lobby/Dashboard loop, assumed the update failed. (...) manually reboot the Opnsense box

The update very likely did not fully commence. Run a health audit (System->Firmware->Status-Run an Audit), I bet it'll come up unclean.

[Kevin G.]

SO, where do I go from here?

Thinking more of a fresh install...but hate the thought of rebuilding everything. Which begs the original questions...

If I fresh install, will my config file bring everything over including Legacy ISC AND Legacy Rules?
OR will a fresh install of 26.7 include the Rule Migration Tool? OR does a fresh 26.7 install have all of it's own basic rules?

[Kevin G.]

Got stuck in a "rebooting now" and returning to the Lobby/Dashboard loop, assumed the update failed. (...) manually reboot the Opnsense box

The update very likely was not complete. Do a health audit (System->Firmware->Status-Run an Audit), I bet it'll come up unclean.

I'll look at this now...thanks.
Wait, does it need to reach the internet to do this?

[drosophila]

Wait, does it need to reach the internet to do this?

It shouldn't need to, but IDK for sure. Try it, if it needs it, it'll stall then fail, hopefully with some descriptive message. :)

Edit: looks like it might do a live lookup, at least it detects version mismatches that stem from a not-yet done update. :\ Unless it uses the last-downloaded database.

If that doesn't come up clean, then the next thing would be to do what this post does (needs terminal and internet access):
https://forum.opnsense.org/index.php?msg=273063

I'll be off updating from 26.7.1_1 to latest now, so hopefully both of us will be running 26.7.2_2 tonight. :)