Messages

1

General Discussion / Re: [Solved] Configuration import verification

« on: Today at 10:27:30 pm »

Mostly because my search for a way to use the config importer under virtualization lead me to this thread
https://forum.opnsense.org/index.php?topic=28020.0
where the OP was trying to do exactly what I was looking for.

My experiments my proxmox predate my experiments with OPNsense by about 1h...
I did not think about adding a 2nd drive. I would also have had to figure out how to copy the config files over.
I just looked at that and it seems reasonable.

I knew how to deal with prebuilt ISOs. Generating one looked easy (until it wasn't because of the entire file name mangling aspect). I learned one thing though: Compare with tools, not visually... And then you have to learn the tools but that's OK.
Looking back, mkisofs would have generated "warnings" in verbose mode during my initial attempt (conversion to 8.3). But it's not generating any without the relaxed-filenames options. That's not nice.

2

General Discussion / Re: Configuration import verification

« on: Today at 01:13:17 am »

Here's the command line that seems to preserve the backup files names:

Code: [Select]

mkisofs -allow-multidot -l -relaxed-filenames -o somefile.iso dirwithslashconf

3

General Discussion / Re: Configuration import verification

« on: Today at 12:26:01 am »

And I had made another mistake...
As can be seen in my previous post, the filenames were still mangled, yet more subtly...
The expected pattern is config-{time}.xml. A dash, not an underscore.
I must have been missing another option when using mkisofs.
I noticed the mismatch when I began to look at the code.

I renamed all the files in place and the GUI now shows the full history.

4

General Discussion / Re: How to Resolve Local Client Names with opnSense

« on: December 02, 2024, 09:31:41 pm »

You can start with Services > Unbound DNS > General : Register ISC DHCP4 Leases
You might need to adjust your devices host names (OS dependent).

If the outcome does not meet your expectations, you can add host overrides:
Services > Unbound DNS > Overrides.
The DHCP dynamic assignments might be stable enough. If they aren't, there's always reservations.

5

General Discussion / Re: OPNSense with APs, Wireless Bridge, VLANs

« on: December 02, 2024, 02:39:31 am »

At this point, you have 2 choices.

#1: remove a physical port from your 3 bridges so you can use it to access the GUI again, untagged.

#2: push forward.
I assume bridge 0 is over ports 0-4 for VLAN 1.
You need to use your switch or your AP to generate tagged VLAN 1 traffic.
It might be easier with the switch:
OPN - Switch - machine.
Configure the switch port connected to OPN as a trunk (and allow at least VLAN 1 tagged. You might need to change the PVID to something != 1).
Configure the switch port connected to the machine as an access port (PVID = 1, VLAN 1 untagged).
With this config, the machine should receive IP from OPN's DHCP.

You can do the same with an AP (OPN - AP - machine) with an SSID associated with VLAN 1.
Hopefully your equipment does not treat VLAN 1 as being special (i.e. native or default).

I'm a little short on equipment to test this myself... But that's what I would try.

6

General Discussion / Re: OPNSense with APs, Wireless Bridge, VLANs

« on: December 01, 2024, 02:53:14 am »

Patrick's recommendation was to dedicate one physical port for management.
You could assign the existing LAN interface to it (or create a new one. you'll need at least one FW rule to open up HTTPS).

If you've created all the bridges already, assigned interfaces that are properly configured with DHCP and would rather not undo some of that, you now have to access the GUI via a VLAN interface, coming into OPN via tagged traffic.

You could get one of your APs to do that. Plug a machine into its Ethernet port with a compatible IP (given how the AP is configured).
Create an SSID for a VLAN of your choice, connect the AP back to OPN. A machine connected to that SSID should get IP from OPN (within the subnet of the VLAN) and be able to access the GUI at the IP address of the gateway of the VLAN (the static IP specified for the VLAN interface).

This said, I'm not sure how you are going to manage your APs afterwards if they don't allow you to specify a management VLAN...

7

General Discussion / Re: Configuration import verification

« on: November 30, 2024, 10:07:40 pm »

That was quicker than expected. I have practice now...
Moving a machine in the same VLAN as the proxmox host helped because I never lost connectivity.

My latest install is up and running. I have not upgraded it just yet.
Before regenerating the ISO, I processed all the backup configs (sed -i s/igc/vtnet/g) and got a fresh copy of the sshd directory (as root so no errors).

All my core settings were imported like last time.
Since I used the old dhcp leases archive, it was obvious that it had been imported properly (some expired leases from a few days ago).
I didn't get an error when I ssh'd back into OPN so the ssh keys were imported fine too.

All the old configs are present in /conf/backup, with proper names this time!
I still have no access to the history in the GUI though.

I noticed the /conf/event_config_changed.json was missing (not handled by the importer code, so that makes sense).
I copied it over and rebooted. No change.
I also checked its content references an existing file in the backup directory.

Code: [Select]

xxx@OPNsense:~ $ cat /conf/event_config_changed.json
{"last_processed_stamp":1732740233.7202}

xxx@OPNsense:~ $ ls /conf/backup
config_1731183347.3.xml         config_1731269020.4566.xml      config_1731531750.1009.xml
...
config_1731207470.4087.xml      config_1731531563.4327.xml      config_1732740233.7202.xml
config_1731207518.8078.xml      config_1731531600.2763.xml
config_1731264024.3773.xml      config_1731531637.3672.xml

System > Configuration > History indicates "no backups available".
System > Configuration > Backups has an empty count and indicates that 2.8M is currently used.

Looks like I'm close...

8

General Discussion / Re: Configuration import verification

« on: November 30, 2024, 12:23:02 am »

Doh!
As I was replying, I doublechecked my steps again and realized that I compared file names against the generated iso, not the original scp'd copy...
And I screwed up generating the iso image, because it didn't preserve the file names (forcing them to 8.3).
That explains the naming scheme difference!
And you're probably referring to opn/conf/event_config_changed.json which points back to the last file.
Given the mangled file names, it probably got quite confused...

But now that I understand this better (I think), I can rerun the entire install.
I'll do a full (conf/backup files) igc->vtnet replacement beforehand, then regen the ISO (with -allow-multidot -l this time).
The safe thing to do would be to run this on a brand new VM, but I could even do it on the current one (as I understand it). I still have the old one to fall back too.

And this time, I'll do that from a machine in the same VLAN as the proxmox machine.
I realized that mistake quickly when I shutdown the VM 
This is all good practice!
Thx all. I'll report back when I do this tomorrow (at the earliest).

9

General Discussion / Re: Configuration import verification

« on: November 29, 2024, 10:44:56 pm »

The source machine was setup a few weeks back (24.7 install media, upgrade to 24.7.7, bunch of changes as I migrated my VLANs over from physical Tp-link router, then a few more as I performed the actual replacement).
All on UFS+passthrough based on online recos.

A few days back, I read a post from @meyergru with different recos and decided to give them a shot (ZFS + bridges to the same NICs).
scp /conf to mgmt PC, replaced igc -> vtnet in config.xml, made iso, uploaded ISO and installed with import.
Same install media, upgraded to 24.7.9 (latest).

I haven't touched the backup settings yet. They are all default.
On the old install, I had full history access.
On the new one, it's not available in the GUI (I realize I would have to replace igc -> vtnet in the files in /conf/backup too for reverts to succeed. Looks doable). The only history that shows is the result on the upgrade.

10

General Discussion / Re: How to set up my network

« on: November 29, 2024, 10:10:32 pm »

I'm not familiar with Orbi routers. I assume you can set it up as a straightforward AP with 1 SSID.
Ditto for the Microtik APs.

Add a VLAN for IOT and another for Guest in OPN.

Code: [Select]

OPN - switch - AP for LAN
                   - LAN devices
                   - AP for IoT
                   - AP for Guest
The port going to AP for IOT needs to be marked as an access port for IOT (PVID = IOT-VLAN, untagged IOT-VLAN).
Ditto for the Guest AP.
The switch port going to OPN needs to allow tagged IOT and Guest traffic.

Devices attached to the IoT AP will get IP from the IOT VLAN subnet (from ISC hosted within OPN).
Ditto with Guest.
They will also reach the Internet (and nothing else on your internal network unless you allow it) via their respective VLAN gateway, also hosted on OPN.

If you had 4 ports on you OPN appliance, you could also dedicate a port to each network.
VLAN allow you to do the same logically and reuse physical ports...

11

General Discussion / Re: Configuration import verification

« on: November 29, 2024, 09:51:13 pm »

So I'm not entirely sure where this leaves me.

I was merely trying to comply with recommendations from @meyergru, hence "converting" from UFS+passthrough to ZFS+bridges.
It's clearly not the previewing/testing upgrade path.
It would fall in the migration path.

I'm a little confused because all the backup files (config history) were copied over the new install.
I suspect the dhcp leases archive was too.
Are you saying they were actually ignored later because the migration code path only cares about the config.xml?
It's not apparent to me how the code is aware of the 2 separate paths (the source drive FS?)...

In any case, given what I am trying to achieve, is there a path forward that preserves history?
Still baffled by the change in naming scheme...

Secondary question:
The previewing/testing upgrade path implies the existence of install media.
This is only available for major releases, right? IOW, the procedure is only useful for major upgrades?

12

General Discussion / Re: How to set up my network

« on: November 28, 2024, 11:27:28 pm »

I think you missed the part where I wrote that you can replicate this behavior with your 3 APs.
The only difference would be that traffic between the switch and the APs wouldn't be tagged.

Edit:
IOW, I'd have 3 virtual APs (on my physical AP) with tagged traffic to the switch.
You'd have 3 physical APs (LAN, Guest  IoT) with untagged traffic to the switch. 2 more switch ports consumed.
The rest of the configuration would be similar (if not identical).

13

General Discussion / Re: Configuration import verification

« on: November 28, 2024, 10:04:54 pm »

And to be more precise on the content of the /conf/backup folder:

Code: [Select]

xxx@OPNsense:/conf/backup $ ls
confi000.xml                    confi00y.xml                    confi01w.xml
confi001.xml                    confi00z.xml                    confi01x.xml
...
confi00u.xml                    confi01s.xml                    confi02q.xml
confi00v.xml                    confi01t.xml                    config-1732751930.5814.xml
confi00w.xml                    confi01u.xml                    config-1732753883.9391.xml
confi00x.xml                    confi01v.xml                    config_1.xml
The naming scheme used to be sequential. All these files are present in the ISO image (& the config_1.xml).
The last 2 files appear to use time in file name. These are the only 2 files I can compare (or revert to) in System > Configuration > History.

I just noticed that the first one has the following note: "Root user reset from console".
It could explain why the earlier content is ignored.
This said, I never explicitly did this beyond the steps outlined earlier.
The last change appears to come from the upgrade I ran ~30 minutes later.

14

General Discussion / Re: Configuration import verification

« on: November 28, 2024, 09:01:27 pm »

I just checked the content of that archive.
It contains the following structure:

Code: [Select]

./var/dhcpd/var/db/
./var/dhcpd/var/db/dhcpd.leases~
./var/dhcpd/var/db/dhcpd6.leases
./var/dhcpd/var/db/dhcpd.leasesThere are clearly more than reservations in there.
I don't really have the patience to do a diff between this content and the now running content (mostly caught up).
But I think I should see expired leases for machines not powered since the reinstall. I do not (all visible leases are more recent).

I'm more concerned about the loss of history.
I personally don't expect to revert back at this point, but it looks like there might be a bug here.

15

General Discussion / Re: Configuration import verification

« on: November 28, 2024, 08:34:26 pm »

I did NOT just restore from the GUI. It's obvious in that case that only the content of the config.xml will be restored.

As indicated in the OP, I used scp to download the entire /conf folder, made an iso from that, mounted the iso to verify its structure and uploaded the iso to proxmox (made available to the VM as a 2nd image).
During install, I triggered the configuration importer (first interactive prompt).

My expectation was indeed that "everything" would be restored (apart from the ssh keys that scp failed to copy for lack of permissions).
My experience is different:
* At least some DHCP leases missing (some machines not powered since the re-install, even with inactive leases checked). The corresponding archive file is present, but I suspect it was recreated by ISC.
Arguably, I just went by the name of the file (/conf/dhcpleas.tgz)...

* Despite a whole bunch of files in /conf/backup, the only history visible in the GUI is 2 entries with files having a different naming scheme.
Arguably scp didn't perserve dates (might have been an option). I don't know if that matters.
The "old" install was running 24.7.7 (IIRC, possibly 24.7..
The ISO used for re-install was 24.7. It was upgraded to 24.7.9 shortly afterwards.