how to recover using an old configuration?

[lmoore]

I had only one plugin installed, so I don't expect that any plugin conflicts would need resolving

Referring to page 6, after you powered on the restored system, did you check the 'Status' page and was the 'Resolve plugin conflicts' button there!?

[(MARLOO)]

@defaultuserfoo 
 I understand your position because I have been in a similar situation before, but the firewall rules are not necessarily lost, and OPNsense has not become obsolete.

In 25.x, firewall rules were managed through the legacy Rules interface.
In 26.1, OPNsense introduced the new MVC/API-based Rules [new] interface. Both systems existed side by side at that point, and migrating the rules was optional.

Before doing anything else, I would first try to recover the current configuration from the old disk using the steps below.

Boot from an OPNsense live/installer USB, with the damaged disk connected and select Shell from the console menu.

First, check whether the old disk is detected:

********************************
camcontrol devlist
gpart show
*****************************

Then look for the ZFS pool:

***********
zpool import
*************

If the pool is shown, import it read-only:

**********************************************************
mkdir /mnt2
zpool import -f -o readonly=on -o altroot=/mnt2 -N zroot
*********************************************************

Then locate and mount the root dataset:

******************************************************
zfs list -r zroot
mount -t zfs -o ro zroot/ROOT/default /mnt2
*********************************************

Copy the configuration:

*********************************************************
cp /mnt2/conf/config.xml /tmp/config.xml
***********************************************************

Finally, plug in a FAT32 USB stick and copy the file to it:


*************************************
mount -t msdosfs /dev/da1s1 /mnt
cp /tmp/config.xml /mnt/config.xml
umount /mnt
***************************************

The USB device name may be different, so check it first with gpart show or camcontrol devlist.



---------------------------------------------------If the disk is not readable-----------------------------------------------------------------------------------

First, check whether the disk is detected at all:

******************************
camcontrol devlist
dmesg | tail -50
gpart show
****************************


If the disk is detected but the ZFS pool will not import, try importing it directly from the ZFS partition:

*******************************************************************************
zpool import
zpool import -d /dev/ada0p3 -f -o readonly=on -o altroot=/mnt2 -N zroot
Check the actual device name with gpart show; it may be ada0p3, da0p3, and so on.
*************************************************************************************

You can also inspect the ZFS labels:

*********************************************************
zdb -l /dev/ada0p3
*********************************************************************

If the pool imports read-only, immediately copy the configuration:

************************************************
zfs list -r zroot
mount -t zfs -o ro zroot/ROOT/default /mnt2
cp /mnt2/conf/config.xml /tmp/config.xml
******************************************

Then copy it to a FAT32 USB stick.

------------------------------------------------If the disk or ZFS pool is corrupted---------------------------------------------------------------------

Check the pool status:

**********************************
zpool import
zpool status
************************************

Try a normal read-only import first

******************************************
mkdir /mnt2
zpool import -f -o readonly=on -o altroot=/mnt2 -N zroot
**************************************************************

If that fails, try importing directly from the ZFS partition

***********************************************************
zpool import -d /dev/ada0p3 -f -o readonly=on -o altroot=/mnt2 -N zroot
********************************************************************************
Check the actual partition name with:

*****************
gpart show
*********************

You can also inspect the ZFS metadata

**************************************
zdb -l /dev/ada0p3
***************************************
If the pool imports read-only, immediately copy the configuration

*********************************************
zfs list -r zroot
mount -t zfs -o ro zroot/ROOT/default /mnt2
cp /mnt2/conf/config.xml /tmp/config.xml
***********************************************

Then copy it to a FAT32 USB stick

-----------------------------------------------------------------------------------------------------------------------------------------------------------
                                                                                                                After recovery

Once you have either recovered the current config.xml from the old disk, I would install OPNsense 25.7 to a new disk or a empty disk installed inside your firewall, then import the latest backup you have recovered.

Do not update for now. First verify that everything works. If it worked before, it should work again after restoring the configuration.

Check the following:

Interface assignments

Installed plugins

Firewall rules

NAT rules

VPN, DNS, DHCP, and any other services you use

Once the system is stable and the interfaces, plugins, firewall rules, and NAT rules are working as expected, immediately create a new backup:

System → Configuration → Backups → Download configuration

Save it outside the firewall, for example on another PC, NAS, or USB drive.

After that, update gradually and test after each step.



Remember that the firewall rules system changed in 26.x. If you import an old configuration with legacy rules, it is normal that they may appear messed up or not editable in the new interface until you migrate them using Firewall → Rules → Migration assistant.

[(MARLOO)]

What's the point of installing an old version of OPNsense to import an old configuration when it can't be upgraded to the current version?  There is no difference between installing the current version and importing the old configuration and installing an old version and importing the old configuration because the result is the same: All the firewall rules are messed up and can't be edited.  They can't be deleted either.

As I said   in the previus post 


There is a point: the firewall rules system has changed.

Firewall → Rules → Migration assistant.
https://docs.opnsense.org/manual/firewall.html

In 25.x, the rules use the legacy Rules system.

In 26.x, OPNsense uses the new MVC/API-based rules system. When you import an old configuration directly into 26.x, the legacy rules are imported, but they need to be migrated to the new system before they can be edited or deleted there.

That is why installing 25.x first is the correct approach: you can restore and verify the old configuration in its original format, then migrate the rules before upgrading to 26.7....

Than you can setup your manual backup,sftp backup with cron job,screenshoot of your plugin and many other things you like.

Regards

[defaultuserfoo]

Migrating the rules does not work

Sorry to hear that.

The notes were written and tested using a configuration file from February 2022 and this process was what worked the smoothest.

Migrating the firewall and NAT rules prior to upgrading to 26.7 allows you to edit them in 26.7.

Well, the reason why it's not working in case might be that I had to take the router to a different location where it's easier to work on it which then required to change interfaces for it to have internet access to install the needed plugin.  It's nonetheless strange because when interfaces are changed or disabled, the importer needs an option to override the error and just import the rule; or the rules need to be adjusted so they can be imported.

That only makes it way more difficult than it already is.  And I still need a solution for the rules.  There are other OPNsense instaces which apparently won't be possible to upgrade when the rules become messed up.

I have to start over :(  The last version before 26.7 seems to be 26.1  I'll install that and see what happens.  The router needs to be fixed a couple days ago.

[defaultuserfoo]

I had only one plugin installed, so I don't expect that any plugin conflicts would need resolving

Referring to page 6, after you powered on the restored system, did you check the 'Status' page and was the 'Resolve plugin conflicts' button there!?

I don't know ...  I'll pay attention to that when I start over.

[defaultuserfoo]

What's the point of installing an old version of OPNsense to import an old configuration when it can't be upgraded to the current version?  There is no difference between installing the current version and importing the old configuration and installing an old version and importing the old configuration because the result is the same: All the firewall rules are messed up and can't be edited.  They can't be deleted either.

As I said   in the previus post 


There is a point: the firewall rules system has changed.

Firewall → Rules → Migration assistant.
https://docs.opnsense.org/manual/firewall.html

In 25.x, the rules use the legacy Rules system.

In 26.x, OPNsense uses the new MVC/API-based rules system. When you import an old configuration directly into 26.x, the legacy rules are imported, but they need to be migrated to the new system before they can be edited or deleted there.

That is why installing 25.x first is the correct approach: you can restore and verify the old configuration in its original format, then migrate the rules before upgrading to 26.7....

Than you can setup your manual backup,sftp backup with cron job,screenshoot of your plugin and many other things you like.

Regards

Oh so I have to use 25.7 and no 26.  I don't want to migrate the rules.  That won't work.  The router needs to be working again since a couple days ago.  I'll try to get it set up with 25.7 and deploy it.

Then I'll have to use the spare machine and set that up and see if the rules can be migrated somehow and either switch the machines or the disks, or migrate the currently broken one.  Having a spare really pays out big time.

[(MARLOO)]

Oh so I have to use 25.7 and no 26.  I don't want to migrate the rules.  That won't work.  The router needs to be working again since a couple days ago.  I'll try to get it set up with 25.7 and deploy it.

Then I'll have to use the spare machine and set that up and see if the rules can be migrated somehow and either switch the machines or the disks, or migrate the currently broken one.  Having a spare really pays out big time.

Do whatever you want, but why do you not want to migrate the rules?

Migrating the rules does not mean changing them. It means validating your existing rules so they work with the new rules system in 26.7.

That way, you can also keep receiving updates.

OPNsense is not obsolete. It is a well-maintained and well-documented system. Major changes are announced in the release notes and discussed in the forum, so it is important to follow those before upgrading an old configuration.

The developers put a lot of work into maintaining the ecosystem and keeping it up to date. Reading the release notes and planning migrations is part of running a firewall, especially after a major architecture change.

Good luck with the recovery. At least now you know why the release notes exist.

[defaultuserfoo]

Oh so I have to use 25.7 and no 26.  I don't want to migrate the rules.  That won't work.  The router needs to be working again since a couple days ago.  I'll try to get it set up with 25.7 and deploy it.

Then I'll have to use the spare machine and set that up and see if the rules can be migrated somehow and either switch the machines or the disks, or migrate the currently broken one.  Having a spare really pays out big time.

Do whatever you want, but why do you not want to migrate the rules?

Migrating the rules does not mean changing them. It means validating your existing rules so they work with the new rules system in 26.7.

No it doesn't.  It means that all the rules need to be redone because migrating them doesn't work.  After importing the rules you get a mess of rules that don't work anymore because of the way things work with the new rules has changed and the old ones are not compatible.

I've done it at home a couple months ago and was left with a ruined firewall which I had to redo.  I was told here that I shouldn't have migrated the rules and that it would be many years before a migration would be required.

I've tried it yesterday when I suddenly found I can't edit the rules anymore after upgrading.  Exporting the old rules and importing them created yet again a total mess with a mixture of immutable rules and non-working rules and maybe some rules weren't imported. The import gets stuck with a popup window having a checkbox and checkmark in it and if you keep clicking on that you get unclear error messages and are left with no idea as to what you're supposed to do now.  You can keep importing the rules over and over but that is futile.

There is no migration.  It does 100% not work.

That way, you can also keep receiving updates.

Only when I redo the whole firewall, and I don't want to do that again.

OPNsense is not obsolete. It is a well-maintained and well-documented system. Major changes are announced in the release notes and discussed in the forum, so it is important to follow those before upgrading an old configuration.

No.  It's not working.  I already said here that this so-called migration was handled badly because there must be a big fat warning in the release notes that your firewall will be broken and that you better wait the however many years before ever trying this so-called migration.  Instead it was made to appear something easy to do.  But it is isn't.

This reminds of the fatal blow Debian struck their users with their stupid brokenarch.  I had been using Debian for about 15 years and then the devs decided to comepletely mess it up, and it was forseeable that the problems would continue to exist in the next release.  So I switched away from Debian.

This is the same crap.  If I knew a good alternative to OPNsense I would switch.

We've already had a discussion about the release notes.  The devs refuse to make good release notes.

The developers put a lot of work into maintaining the ecosystem and keeping it up to date. Reading the release notes and planning migrations is part of running a firewall, especially after a major architecture change.

Good luck with the recovery. At least now you know why the release notes exist.

The relase notes suck.  Look up the discussion about it we had.

It's not about the release notes anyway.  The problem is that there is no way to update the OPNsense instances anymore because doing that will break the firewall rules and the firewall needs to be completely redone.  That makes OPNsense obsolete.

Maybe I'll find an alternative.  If I have to start over from scratch I can as well switch to something else.

[defaultuserfoo]

I had only one plugin installed, so I don't expect that any plugin conflicts would need resolving

Referring to page 6, after you powered on the restored system, did you check the 'Status' page and was the 'Resolve plugin conflicts' button there!?

I don't know ...  I'll pay attention to that when I start over.

I've installed the plugin that will be needed and there is no button to resolve plugin conflicts.  I'll check again after importing the old configuration.

[defaultuserfoo]

Than you can setup your manual backup,sftp backup with cron job,screenshoot of your plugin and many other things you like.

Oh, are you suggesting to log in on the console and set up a cron job to use the ftp client to make a copy of /conf/config.xml?

Is there a better way?  I could as well use scp maybe.  Unfortunately, rsync is not installed.  Or is there a way through the web interface?

[defaultuserfoo]

I had only one plugin installed, so I don't expect that any plugin conflicts would need resolving

Referring to page 6, after you powered on the restored system, did you check the 'Status' page and was the 'Resolve plugin conflicts' button there!?

I don't know ...  I'll pay attention to that when I start over.

I've installed the plugin that will be needed and there is no button to resolve plugin conflicts.  I'll check again after importing the old configuration.

There were no conflicts after importing the configuration.  But the plugin (os-maltrail) shows as orphaned.

[nero355]

No. It's not working.

Instead it was made to appear something easy to do.  But it is isn't.

This reminds of the fatal blow Debian struck their users with their stupid brokenarch.  I had been using Debian for about 15 years and then the devs decided to comepletely mess it up, and it was forseeable that the problems would continue to exist in the next release.  So I switched away from Debian.

This is the same crap.  If I knew a good alternative to OPNsense I would switch.

As someone who is using both Debian and OPNsense I totally disagree with you :)

We've already had a discussion about the release notes.  The devs refuse to make good release notes.

NOFI but IMHO it's you and not the software...

Maybe try something OpenWRT based or simply buy stuff like TP-Link Omada or Ubiquiti UniFi or HPE Aruba ?!



Good luck! :)