[HOWTO] Virtual Cold Spare for a Physical OPNsense Using Proxmox

[meyergru]

I used to keep a second, physical OPNsense as a cold spare, just in case I messed up the primary firewall or its hardware failed. There are two annoyances with that approach: keeping the spare updated, and getting the current production configuration onto it precisely when the primary box may no longer be available.

Also, hardware has become expensive. A small China box suitable for the job used to cost around €300; similar hardware can now easily be closer to €600, depending on the model.

So I thought: why not use a virtual cold spare on an existing Proxmox host instead?

The idea is straightforward. The spare OPNsense normally runs as an ordinary maintenance client behind the real firewall, with its own IP and MAC address. You can boot it, update it and test it without taking down your network. If the primary firewall fails, shut it down, switch the VM to emergency mode, and optionally import a current physical OPNsense config.xml. The VM then uses the original interface names, MAC addresses and network connections.

This is not HA, CARP, pfsync or automatic failover. It is a manually activated cold spare, designed to be comparatively simple and cheap.

Source code and configuration examples: https://github.com/meyergru/opnsense-spare

Current status: Automated tests pass, but the generalized, single-hook implementation has not yet completed a full Proxmox/OPNsense boot, WAN cutover and failback integration test. Please test in an isolated environment before relying on it as a recovery procedure.

Prerequisites

• A Proxmox VE host with sufficient spare CPU/RAM/storage to run an OPNsense VM, ideally on a machine independent of the physical firewall.
• A physical OPNsense and a reasonably recent backup of its config.xml, accessible even if the firewall is completely dead.
• A Layer-2 path from each physical firewall network to the corresponding virtual NIC. This may use dedicated PVE NICs/bridges, VLANs on an existing trunk, or a combination. Neither UniFi nor VLANs are mandatory.
• The physical firewall's interface names and MAC addresses, recorded in advance. Access to the Proxmox console is essential while setting this up.
• An OPNsense guest installed with VirtIO NICs (vtnet0, vtnet1, etc.); Python 3 and xorriso on Proxmox.

Step by step

1. Install OPNsense in a VM

For general guidance on installing and tuning OPNsense under Proxmox, see my earlier OPNsense under virtualisation (Proxmox et al.). The instructions below concentrate on the additional requirements for the cold-spare setup.

Create a normal OPNsense VM on Proxmox. For the firmware, select OVMF (UEFI) and add an EFI Disk (VM -> Options -> BIOS, and VM -> Hardware -> Add -> EFI Disk). The physical OPNsense may boot via legacy BIOS; that does not matter because we import its configuration, not its boot disk. The firmware type is not part of config.xml.

Give the VM as many VirtIO network adapters as the physical firewall requires. Keep their net0, net1, etc. ordering stable. Disable automatic VM startup. Set up the necessary Linux bridges first.

Add a serial console from the outset, even when using the graphical installer and VGA as the normal console. On the Proxmox host, with the VM stopped:

Code Select Expand

qm set <VMID> --serial0 socket

Alternatively, use VM -> Hardware -> Add -> Serial Port, select port 0 and socket. Leave a VGA display available for the normal noVNC console. On the host, connect to the serial port with:

Code Select Expand

qm terminal <VMID>

If you prefer a serial console in the browser, change VM -> Hardware -> Display to Serial terminal 0, then open the VM's Console tab (xterm.js). This selects serial output instead of the normal VGA/noVNC view. For actual serial output from OPNsense, go to System -> Settings -> Administration -> Console and enable Serial console as the primary or secondary console (usually 115200 baud). The VM port by itself does not redirect the guest's output. Test both console paths, including after an emergency configuration import: the imported settings may change the primary console.

Install OPNsense normally and initially use it as a maintenance LAN client: separate IP, separate MAC, default gateway via your existing physical firewall. It must not start competing DHCP, router advertisements or other production gateway services on that network. You should be able to reach its GUI from the LAN.

Why are interface names important? Your physical firewall's config.xml may refer to ix1, igc3, etc. In a VirtIO VM the devices are normally called vtnet0, vtnet1. Restoring the configuration without addressing this difference will not reproduce the physical interface assignments.

Rather than editing the imported XML, the spare changes the FreeBSD interface names very early in the guest's boot process, before OPNsense configures its network:

Code Select Expand

Proxmox net0 -> vtnet0 -> renamed to ix1  -> config.xml uses ix1
Proxmox net1 -> vtnet1 -> renamed to igc3 -> config.xml uses igc3

This changes the name, not the underlying VirtIO driver. Hardware-specific tunables, offloading or IPS configurations can still require adjustment.

Why must Maintenance and Emergency use different MAC addresses?

The same VM plays two very different roles:

• Maintenance means coexistence. The physical OPNsense remains online while the VM runs as an ordinary LAN client. Therefore the VM needs its own LAN IP address AND its own MAC address. A different IP alone is not enough: two active NICs with the same MAC on the same Layer-2 network cause ambiguous switch forwarding (MAC-table flapping), even if their IP addresses differ. Conversely, separate MACs do not fix an IP conflict if both firewalls use the same LAN address. Any other VM NICs that would reach production networks remain disconnected in this mode.
• Emergency means replacement. Once the physical firewall is powered off or completely isolated, the VM assumes the original interface names, the original production IP addresses (from the imported/saved OPNsense configuration), and the original MAC address for each corresponding physical NIC. This makes the replacement look like the old firewall to neighboring devices. It avoids unnecessary changes to DHCP reservations and ISP MAC-based assignments, and minimizes ARP/neighbor-cache disruption. Matching MACs are not intrinsically required by all networks, but cloning them is the most transparent choice for a cold spare.

So we deliberately change the VM's MAC addresses when switching modes. The Proxmox controller applies the MACs specified in the two JSON network profiles; the OPNsense guest does not modify them. Retrieve the production values from ifconfig -a on the physical firewall (look for ether), and generate a unique locally administered MAC for the maintenance client. Never enable the Emergency links while the original firewall is still connected. For a test alongside the physical firewall, use --link-down so all Emergency VM links remain disconnected.

2. Install the guest 90-opnsense-spare hook

Get 90-opnsense-spare from the repository and copy it into the VM, then run:

Code Select Expand

install -m 755 90-opnsense-spare \
  /usr/local/etc/rc.syshook.d/import/90-opnsense-spare
/usr/local/etc/rc.syshook.d/import/90-opnsense-spare init

init saves the current maintenance configuration as the initial maintenance profile. The guest's one import hook then does two things on every boot: read the mode request from the ISO and rename the NICs. It also selects/imports the requested configuration when necessary. No SSH or working network connection is needed for an import, even when all VM NICs have link_down=1.

Note that the initial maintenance configuration may still assign vtnet0. After the first boot with the new hook, that NIC will be called ix1 (or whatever you configured). Use the Proxmox console to correct the maintenance interface assignment/IP/gateway, then verify a second boot. Do not depend on remote access for this first transition.

3. Install the controller on Proxmox

Obtain opnsense-spare and opnsense-spare.json.example from the same repository. On the PVE host:

Code Select Expand

apt install xorriso
install -m 755 opnsense-spare /usr/local/sbin/opnsense-spare
install -m 600 opnsense-spare.json.example /etc/opnsense-spare.json
editor /etc/opnsense-spare.json

The controller changes the VM's Proxmox NIC configuration and attaches a small read-only ISO. That ISO contains the requested mode and interface mapping, and only when explicitly requested a new physical config.xml. The guest performs the actual mode switch during early boot. The ISO has an internal unique request ID, so an ordinary reboot cannot keep reimporting an old config.xml and destroy newer changes.

4. Create and verify the configuration

The JSON describes your VM ID, the network profiles for maintenance and emergency, the mapping of VirtIO names to the original physical names, and the optional SSH/export settings. These are the important parts of the two-NIC example (the actual example file also contains storage, SSH and transfer settings):

Code Select Expand

"interface_names": {
  "vtnet0": "ix1",
  "vtnet1": "igc3"
},
"maintenance_client_nic": "net0",
"networks": {
  "maintenance": {
    "net0": "virtio=02:00:00:00:10:01,bridge=vmbr0,tag=10,firewall=0",
    "net1": "virtio=02:00:00:00:10:02,bridge=vmbr0,link_down=1,firewall=0"
  },
  "emergency": {
    "net0": "virtio=02:00:00:00:20:01,bridge=vmbr0,tag=4094,trunks=1;7;10;52;55;56,firewall=0",
    "net1": "virtio=02:00:00:00:20:02,bridge=vmbr0,tag=41,trunks=40,firewall=0"
  }
}

IMPORTANT: The 02:00:00:... MAC addresses above are illustrative placeholders, not usable defaults. Set networks.emergency.netN to the corresponding physical interface's real MAC and networks.maintenance.net0 to an independent, unique MAC. The maintenance OPNsense configuration also requires an independent LAN IP address; the emergency configuration uses the original firewall's addresses. Replace bridge names, VLAN IDs and paths to match your topology. The script cannot discover them automatically.

Check that the PVE VM already has the expected number of NICs, and that your chosen bridges exist. Then, with the VM stopped, run:

Code Select Expand

opnsense-spare maintenance --dry-run
opnsense-spare maintenance

Start the VM when prompted, fix/check the interface assignment from the PVE console, and verify management connectivity and a repeat boot.

For the first emergency test, keep the real firewall running and disconnect all virtual NIC links:

Code Select Expand

opnsense-spare emergency --link-down \
  --transfer /root/opnsense-transfer/config.xml

Confirm the import and inspect the resulting OPNsense through the PVE console. The first emergency boot needs a config.xml; later you may decline reimporting and reuse the internally saved emergency profile. With --link-down, neither SSH nor any other guest network connectivity is needed.

For a real cutover, shut down the physical firewall and move any necessary cables first. Then use opnsense-spare emergency --link-up, following its prompts. When the physical firewall is repaired, shut down the VM and switch it back to maintenance. An optional SSH export makes the last emergency configuration available on the PVE host for restoring to the physical box.

5. Different network topologies: tag, trunks and physical ports

This is the part you must adapt to your own network. The project is switch-vendor neutral, and the physical OPNsense does not even need to use VLANs.

A PVE VirtIO NIC definition such as:

Code Select Expand

virtio=<MAC>,bridge=vmbr0,tag=4094,trunks=1;10;50

means:

• bridge=vmbr0 is the PVE Linux bridge carrying this VM NIC.
• tag=4094 selects VLAN 4094 as the VM port's untagged/native VLAN (PVID). In this case it is intentionally unused, to avoid putting production untagged traffic onto a live network.
• trunks=1;10;50 permits those VLANs as tagged frames to/from the guest.

My UniFi / VLAN 1 case: My PVE host uses native/untagged VLAN 1 for management on its UniFi uplink, while the physical OPNsense expects VLAN 1 tagged on its LAN trunk. The emergency VM needs to see that VLAN tagged, too. Putting the VM port on a dummy/native VLAN 4094 and allowing VLAN 1 in trunks provides this arrangement on a properly configured VLAN-aware PVE bridge. The physical and PVE uplink port profiles are not necessarily identical. Verify ingress and egress 802.1Q behavior rather than assuming that VLAN 1 is treated the same everywhere.

For the WAN in my particular setup, VLAN 40 carries PPPoE and VLAN 41 transports the ONT's untagged management network across the switch. That explains:

Code Select Expand

virtio=<physical-WAN-MAC>,bridge=vmbr0,tag=41,trunks=40

It is not a universal WAN requirement. If your ISP supplies an untagged DHCP WAN and your PVE host has a spare physical Ethernet adapter, create a separate bridge (say vmbrWAN) connected to that adapter, without assigning a host IP, and use:

Code Select Expand

virtio=<physical-WAN-MAC>,bridge=vmbrWAN

Move the WAN cable to that adapter in an emergency. No WAN transport VLAN or ONT management VLAN is needed.

For a physical firewall with igc0 = untagged LAN, igc1 = tagged LAN VLANs, igc3 = untagged WAN/DHCP, use three VM NICs, three original MACs, and a mapping of vtnet0=igc0, vtnet1=igc1, vtnet2=igc3. The repository includes opnsense-spare-three-nic.json.example. Your PVE bridges can be backed by separate physical NICs or by a suitably configured switch/trunk.

6. Side notes and caveats

SSH agent versus a key file: The initial import does not need SSH. The optional return export is performed later, when maintenance mode is running and reachable over the LAN. You can specify an SSH identity file in the JSON or use "ssh_identity": null and connect to the PVE host with agent forwarding, e.g. ssh -A root@pve-host. The public key must be authorized for the maintenance OPNsense user. Do not disable host-key verification.

UEFI/EFI bootloader: As described above, I recommend OVMF/UEFI for the VM, independently of how the physical firewall boots. I ran into a separate problem after importing a newer physical OPNsense configuration into an older VM installation: a reboot loop during EFI framebuffer/console initialization, before the import hook was even relevant. In that case, updating the VM's EFI loader from its matching OPNsense/FreeBSD installation fixed it. If you encounter this, test serial-console boot and check the ESP and its EFI/BOOT/BOOTX64.EFI and EFI/FreeBSD/loader.efi copies against the installed /boot/loader.efi. Back up the EFI files and identify the correct ESP before changing anything. This is not part of the normal cold-spare procedure and must not be applied blindly.

CPU microcode: If the physical OPNsense uses a CPU microcode plugin, its presence in the configuration is not a problem for the cold spare. The import transfers only the XML, not any microcode binaries. Even if the plugin is installed in the guest, it cannot update the physical CPU through the virtual machine; CPU microcode updates are handled by the Proxmox host. There is therefore no need to remove the plugin from the imported configuration or to treat it as a migration hazard. Installing it separately in the guest serves no useful purpose.

Cold standby, not magic: NIC names and MACs can be reproduced, but physical NIC-specific tunables, plugins, IPS/offloading, interface offload settings and WAN-specific constraints may not transfer cleanly. Test VLANs, DHCP/PPPoE, firewall rules and routing before relying on the VM. Never operate the physical firewall and the emergency VM simultaneously with the same production addresses. Proxmox itself must remain reachable without depending on the failed firewall in an unresolvable way.

Security and disclaimer

I generally recommend not installing untrusted software on security-critical systems such as OPNsense or Proxmox. This project is deliberately straightforward and the source is available for inspection. The guest hook essentially selects/imports a configuration and renames interfaces early during boot. The PVE controller manages VM NIC profiles, packages the mode/configuration files into an ISO, and optionally handles a configuration export. There is no permanently running synchronization agent. The scripts are small enough to audit compared with a full HA stack, and they should not normally need frequent updates.

That said, read the scripts before running them. The ISO and exported config.xml may contain passwords, keys and other firewall secrets: protect them accordingly. A successful unit test is not a substitute for a real disaster-recovery drill.

Everything is provided as-is, without warranties or guarantees of any kind. You are responsible for validating the solution on your own hardware and topology.

This suits my needs - you have to decide if it suits yours.

[(MARLOO)]

Great project

One implementation: since the WAN cutover already requires physically moving the cable to a spare Proxmox NIC, a small helper script could watch for link-up on that port and then automatically run opnsense-spare emergency --link-up --transfer /path/to/latest/config.xml, using the latest nightly config.xml backup.

With a safety delay or confirmation prompt, this would turn the manual cold spare into a semi-automatic takeover: just move the WAN cable and the VM takes over with the latest configuration.

Do you think this could be an optional "auto-cutover" feature?

[meyergru]

No, because if you read the documentation closely, you'll see that the design deliberately abstracts from the specific network topology. There isn't necessarily a dedicated WAN port on the Proxmox host to monitor. In my case, for example, the ONT is connected to a switch port.

More importantly, a link-up event does not tell you whether the physical firewall has been shut down. Accidentally connecting something to that port could activate the emergency instance while the physical firewall is still running, resulting in conflicting IP and MAC addresses. Besides, monitoring ports on PVE is far more intrusive than what I would want security-wise.

The intention is to keep the cold spare in maintenance mode during normal operation, so it can be updated regularly and independently of the physical firewall. Switching to emergency mode is a deliberate manual operation after ensuring that the physical firewall is no longer active.

This is a **cold spare**, not an automatic failover solution. If you want automatic failover, use a proper HA setup.

[nero355]

In that case, updating the VM's EFI loader from its matching OPNsense/FreeBSD installation fixed it. If you encounter this, test serial-console boot and check the ESP and its EFI/BOOT/BOOTX64.EFI and EFI/FreeBSD/loader.efi copies against the installed /boot/loader.efi. Back up the EFI files and identify the correct ESP before changing anything. This is not part of the normal cold-spare procedure and must not be applied blindly.

I am almost done with documenting everything about updating the OPNsense/FreeBSD Boot Loader based on the information written by many here on the forum and some official documentation too.

Once it's publicly available on GitHUB I will open a topic in the HowTo sub-forum about it :)