opnsense-update killed my machine?

[Hektor]

Hello,

I'm using OPNsense as a Hyper-V Server 2012 R2 VM since some weeks but it's running on a test machine so far - fortunately!

I was running the following version:
OPNsense 15.7.25-amd64
FreeBSD 10.1-RELEASE-p27
OpenSSL 1.0.2e 3 Dec 2015

I just ran opnsense-update in the hope to get the 16.1 version at the console and did a reboot aas suggested after the update.

Now it doesn't mount the root partition anymore and says:
Mounting from ufs:/dev/ada0s1a failed with error 19

... some more generell lines ...

mountroot>

So what can I do now and why did this happen while running the opnsense-update command?

There is nothing fancy installed and no advanced configuration. No Captiva, no IDS etc. Only simple NAT and OpenVPN tests with the wizard.

If everything fails:
Can I simply extract the /conf/config.xml and import it into a freshly installed OPNsense 16.1 VM?

[weust]

The config.xml part I cant help you with, the /dev/ada thing I can.
FreeBSD 10.2 doesn't support /dev/ada, but only /dev/da.
IIRC there is a line at the bottom of that error message telling you how to continue.
Something like boot or mount /dev/xxx? Use /dev/da0s1a and it will continue to boot.

One note: Most likely you won't have anymore traffic going on though the firewall/router.
There is something odd going on in either base or kernel, or a combination, that makes pretty much every packet get dropped.
Fitch is working on this, and I've tested several scenarios with test build for him.
No real luck so far.

Best is to stay at 15.7.25 for now.
And hit us up on IRC.

[Hektor]

Hi,

thanks for your fast reply.

I already tried the ? command and it showed me some devices. I just tried the /dev/da0s1a and it booted!

Very good to get some confirmation from you so I did a vi /etc/fstab and removed the "evil a" and a reboot just worked :-)

Bad news is:
Seems that you were right with the network issues - where on IRC are you?

[franco]

The disk issue was noted in the EOL message in the 15.7 firmware screen.

The networking issue seems to be a regression in 10.2 that has not made the errata notes yet, but it seems we have found a promising bug candidate:

https://github.com/opnsense/core/issues/734

[Hektor]

Well ... seems that I didn't see the message about the /dev/ada device. To be honest it was a "ok, let's do the update and reboot, the "installer" would tell me and ask for confirmation for such breaking changes" upgrade. And in the end it's still a test and playground machine for some days so I don't need to take much care of it yet :-)

Thanks for the link regarding the Hyper-V issue. Seems that I can't download the test kernel provided by "fichtner" to test it. Local network on the OPNsense box seems to be dead, too.

Will give it a try tomorrow if time permits and if I can get the kernel to the machine somehow. How can I get the test-kernel onto the machine without network access? ;-) Didn't try ssh yet so...

[franco]

"fichtner" would be me, but don't trust that... every update set is signed with the OPNsense key so you'll see when it's not trusted it won't apply.

We pondered options about the message or a more intrusive approach, but decided to not force rewriting /etc/fstab for simplicity and sanity. Who knows what could have gone wrong. Anyway, what counts is solving the 10.2 regression.

The best course of action would be to verify with a 15.7.25 installation, using the opnsense-upgrade command to get to the test operating system and reboot. If the network comes back up normally, the patching worked...

# opnsense-update -bkr 16.1-hyperv && /usr/local/etc/rc.reboot

If the patch works, 16.1.2 will have the fix and you'll be able to safely upgrade from 15.7.25.

[weust]

Hektor, I wasn't on IRC today. Sorry, man!
I'm usually online during the day while at work. In the evenings I tend to game when I can :-)

Glad to see this annoying bug come to an end.
Going to test it tomorrow evening if I can, on a fresh 16.1 installation.

[franco]

We talked on IRC nonetheless and discussed how to upgrade to the test version properly.

[weust]

Good to hear :-)