[Solved] Configuration import verification

[EricPerl]

So I'm not entirely sure where this leaves me.

I was merely trying to comply with recommendations from @meyergru, hence "converting" from UFS+passthrough to ZFS+bridges.
It's clearly not the previewing/testing upgrade path.
It would fall in the migration path.

I'm a little confused because all the backup files (config history) were copied over the new install.
I suspect the dhcp leases archive was too.
Are you saying they were actually ignored later because the migration code path only cares about the config.xml?
It's not apparent to me how the code is aware of the 2 separate paths (the source drive FS?)...

In any case, given what I am trying to achieve, is there a path forward that preserves history?
Still baffled by the change in naming scheme...

Secondary question:
The previewing/testing upgrade path implies the existence of install media.
This is only available for major releases, right? IOW, the procedure is only useful for major upgrades?

[franco]

If the backups were set to back up they are restored in the config-import-install case. It depends on how old the installation was WRT default backup settings and if they were manually altered.

If you copied the /conf folder from an older install to a newer everything is retained but not used depending on periodic backup settings (again).


Cheers,
Franco

[EricPerl]

The source machine was setup a few weeks back (24.7 install media, upgrade to 24.7.7, bunch of changes as I migrated my VLANs over from physical Tp-link router, then a few more as I performed the actual replacement).
All on UFS+passthrough based on online recos.

A few days back, I read a post from @meyergru with different recos and decided to give them a shot (ZFS + bridges to the same NICs).
scp /conf to mgmt PC, replaced igc -> vtnet in config.xml, made iso, uploaded ISO and installed with import.
Same install media, upgraded to 24.7.9 (latest).

I haven't touched the backup settings yet. They are all default.
On the old install, I had full history access.
On the new one, it's not available in the GUI (I realize I would have to replace igc -> vtnet in the files in /conf/backup too for reverts to succeed. Looks doable). The only history that shows is the result on the upgrade.

[franco]

Ok, the JSON file may be interfering, but it's not my area of expertise. Just to be sure the old backups are in /conf/backup ?


Cheers,
Franco

[EricPerl]

Doh!
As I was replying, I doublechecked my steps again and realized that I compared file names against the generated iso, not the original scp'd copy...
And I screwed up generating the iso image, because it didn't preserve the file names (forcing them to 8.3).
That explains the naming scheme difference!
And you're probably referring to opn/conf/event_config_changed.json which points back to the last file.
Given the mangled file names, it probably got quite confused...

But now that I understand this better (I think), I can rerun the entire install.
I'll do a full (conf/backup files) igc->vtnet replacement beforehand, then regen the ISO (with -allow-multidot -l this time).
The safe thing to do would be to run this on a brand new VM, but I could even do it on the current one (as I understand it). I still have the old one to fall back too.

And this time, I'll do that from a machine in the same VLAN as the proxmox machine.
I realized that mistake quickly when I shutdown the VM 
This is all good practice!
Thx all. I'll report back when I do this tomorrow (at the earliest).

[EricPerl]

That was quicker than expected. I have practice now...
Moving a machine in the same VLAN as the proxmox host helped because I never lost connectivity.

My latest install is up and running. I have not upgraded it just yet.
Before regenerating the ISO, I processed all the backup configs (sed -i s/igc/vtnet/g) and got a fresh copy of the sshd directory (as root so no errors).

All my core settings were imported like last time.
Since I used the old dhcp leases archive, it was obvious that it had been imported properly (some expired leases from a few days ago).
I didn't get an error when I ssh'd back into OPN so the ssh keys were imported fine too.

All the old configs are present in /conf/backup, with proper names this time!
I still have no access to the history in the GUI though.

I noticed the /conf/event_config_changed.json was missing (not handled by the importer code, so that makes sense).
I copied it over and rebooted. No change.
I also checked its content references an existing file in the backup directory.

Code: [Select]

xxx@OPNsense:~ $ cat /conf/event_config_changed.json
{"last_processed_stamp":1732740233.7202}

xxx@OPNsense:~ $ ls /conf/backup
config_1731183347.3.xml         config_1731269020.4566.xml      config_1731531750.1009.xml
...
config_1731207470.4087.xml      config_1731531563.4327.xml      config_1732740233.7202.xml
config_1731207518.8078.xml      config_1731531600.2763.xml
config_1731264024.3773.xml      config_1731531637.3672.xml

System > Configuration > History indicates "no backups available".
System > Configuration > Backups has an empty count and indicates that 2.8M is currently used.

Looks like I'm close...

[EricPerl]

And I had made another mistake...
As can be seen in my previous post, the filenames were still mangled, yet more subtly...
The expected pattern is config-{time}.xml. A dash, not an underscore.
I must have been missing another option when using mkisofs.
I noticed the mismatch when I began to look at the code.

I renamed all the files in place and the GUI now shows the full history.

[EricPerl]

Here's the command line that seems to preserve the backup files names:

Code: [Select]

mkisofs -allow-multidot -l -relaxed-filenames -o somefile.iso dirwithslashconf

[franco]

Thanks, that's very useful. Actually a bit of a tricky case the whole import from ISO case.


Cheers,
Franco

[Patrick M. Hausen]

@EricPerl why did you use an ISO in the first place? It's a virtual machine, right? You could have attached a second virtual hard disk with UFS mimicking a USB drive ...