Backup FW ignoring CARP for WireGuard Tunnel

[davorin]

Weird behaviour of our backup FW running 25.7.6 where WireGuard tunnel is ignoring the WAN CARP state.

The master FW shows no log entries and it stays always the master for the WireGuard tunnel.
The backup FW shows in the WireGuard logs permanently a state change of the WAN CARP and takes over the WireGuard tunnel, although the state of the WAN interface is backup.

The other side of the tunnel is also a HA setup running 26.1.2, but there is no flapping of the tunnel on the backup FW.

Anyone else seing this odd behaviour?

Problem is that I had to disable WireGuard instances and HA syncing of WireGuard configuration.

[davorin]

Did now a virtualized test setup with a master/backup running CARP on WAN and LAN and a WG tunnel to a third OPNSense installation...

Tunnel runs fine on master FW, but as soon I change high availability settings to include WireGuard for syncing, the backup FW immediately takes over and becomes the master. After around 70 seconds the backup FW redraws and all is fine again.

Code Select Expand

2026-03-09T12:23:30 Notice kernel <6>[2133] wg0: link state changed to DOWN
2026-03-09T12:23:30 Notice wireguard Wireguard configure event instance Test (wg0) vhid: 10 carp: BACKUP interface: down
2026-03-09T12:23:30 Notice wireguard wireguard instance Test (wg0) switching to DOWN
2026-03-09T12:23:30 Notice wireguard Wireguard configure event instance Test (wg0) vhid: 10 carp: BACKUP interface: up
2026-03-09T12:22:24 Notice wireguard Wireguard configure event instance Test (wg0) vhid: 10 carp: MASTER interface: up
2026-03-09T12:22:24 Notice kernel <6>[2068] wg0: link state changed to UP
2026-03-09T12:22:24 Notice wireguard wireguard instance Test (wg0) switching to UP
2026-03-09T12:22:24 Notice wireguard Wireguard configure event instance Test (wg0) vhid: 10 carp: MASTER interface: down
2026-03-09T12:21:30 Notice kernel <6>[2014] wg0: link state changed to DOWN
2026-03-09T12:21:30 Notice kernel <6>[2014] wg0: link state changed to UP
2026-03-09T12:21:30 Notice wireguard wireguard instance Test (wg0) started
2026-03-09T12:21:30 Notice wireguard /usr/local/opnsense/scripts/wireguard/wg-service-control.php: plugins_configure monitor (execute task : dpinger_configure_do(,[]))
2026-03-09T12:21:30 Notice wireguard /usr/local/opnsense/scripts/wireguard/wg-service-control.php: plugins_configure monitor (,[])
2026-03-09T12:21:30 Notice wireguard /usr/local/opnsense/scripts/wireguard/wg-service-control.php: ROUTING: entering configure using opt2
2026-03-09T12:21:30 Notice wireguard wireguard instance Test (wg0) stopped


[davorin]

Being a little further...master FW was blocking Multicast on pfSync interface from backup FW....

But still every time I change settings for System->HA, I see in the WG logs on the backup FW:

Code Select Expand

2026-03-09T15:31:14    Notice    wireguard     Wireguard configure event instance Office-SiteA (wg2) vhid: 50 carp: BACKUP interface: down
2026-03-09T15:31:14    Notice    wireguard     Wireguard configure event instance Office-SiteB (wg1) vhid: 50 carp: BACKUP interface: down
2026-03-09T15:31:14    Notice    wireguard     Wireguard configure event instance Office-SiteC (wg0) vhid: 50 carp: BACKUP interface: down
2026-03-09T15:31:14    Notice    wireguard     Wireguard configure event instance Office-SiteA (wg2) vhid: 50 carp: BACKUP interface: down
2026-03-09T15:31:14    Notice    wireguard     Wireguard configure event instance Office-SiteB (wg1) vhid: 50 carp: BACKUP interface: down
2026-03-09T15:31:14    Notice    wireguard     Wireguard configure event instance Office-SiteC (wg0) vhid: 50 carp: BACKUP interface: down
2026-03-09T15:31:14    Notice    wireguard     Wireguard configure event instance Office-SiteA (wg2) vhid: 50 carp: BACKUP interface: down
2026-03-09T15:31:14    Notice    wireguard     Wireguard configure event instance Office-SiteB (wg1) vhid: 50 carp: BACKUP interface: down
2026-03-09T15:31:14    Notice    wireguard     Wireguard configure event instance Office-SiteC (wg0) vhid: 50 carp: BACKUP interface: down
2026-03-09T15:31:13    Notice    wireguard     Wireguard configure event instance Office-SiteA (wg2) vhid: 50 carp: BACKUP interface: down
2026-03-09T15:31:13    Notice    wireguard     Wireguard configure event instance Office-SiteB (wg1) vhid: 50 carp: BACKUP interface: down
2026-03-09T15:31:13    Notice    wireguard     Wireguard configure event instance Office-SiteC (wg0) vhid: 50 carp: BACKUP interface: down
2026-03-09T15:31:13    Notice    wireguard     Wireguard configure event instance Office-SiteA (wg2) vhid: 50 carp: BACKUP interface: down
2026-03-09T15:31:13    Notice    wireguard     Wireguard configure event instance Office-SiteB (wg1) vhid: 50 carp: BACKUP interface: down
2026-03-09T15:31:13    Notice    wireguard     Wireguard configure event instance Office-SiteC (wg0) vhid: 50 carp: BACKUP interface: down
2026-03-09T15:31:13    Notice    wireguard     Wireguard configure event instance Office-SiteA (wg2) vhid: 50 carp: BACKUP interface: down
2026-03-09T15:31:13    Notice    wireguard     Wireguard configure event instance Office-SiteB (wg1) vhid: 50 carp: BACKUP interface: down
2026-03-09T15:31:13    Notice    wireguard     Wireguard configure event instance Office-SiteC (wg0) vhid: 50 carp: BACKUP interface: down
2026-03-09T15:31:13    Notice    wireguard     Wireguard configure event instance Office-SiteA (wg2) vhid: 50 carp: BACKUP interface: down
2026-03-09T15:31:13    Notice    wireguard     Wireguard configure event instance Office-SiteB (wg1) vhid: 50 carp: BACKUP interface: down
2026-03-09T15:31:13    Notice    wireguard     Wireguard configure event instance Office-SiteC (wg0) vhid: 50 carp: BACKUP interface: down
2026-03-09T15:31:13    Notice    wireguard     Wireguard configure event instance Office-SiteA (wg2) vhid: 50 carp: BACKUP interface: down
2026-03-09T15:31:13    Notice    wireguard     Wireguard configure event instance Office-SiteB (wg1) vhid: 50 carp: BACKUP interface: down
2026-03-09T15:31:13    Notice    wireguard     Wireguard configure event instance Office-SiteC (wg0) vhid: 50 carp: BACKUP interface: down
2026-03-09T15:31:12    Notice    wireguard     Wireguard configure event instance Office-SiteA (wg2) vhid: 50 carp: BACKUP interface: down
2026-03-09T15:31:12    Notice    wireguard     Wireguard configure event instance Office-SiteB (wg1) vhid: 50 carp: BACKUP interface: down
2026-03-09T15:31:12    Notice    wireguard     Wireguard configure event instance Office-SiteC (wg0) vhid: 50 carp: BACKUP interface: down
2026-03-09T15:31:12    Notice    wireguard     Wireguard configure event instance Office-SiteA (wg2) vhid: 50 carp: BACKUP interface: down
2026-03-09T15:31:12    Notice    wireguard     Wireguard configure event instance Office-SiteB (wg1) vhid: 50 carp: BACKUP interface: down
2026-03-09T15:31:12    Notice    wireguard     Wireguard configure event instance Office-SiteC (wg0) vhid: 50 carp: BACKUP interface: down

Is this the expected behaviour?

Because when I change HA settings the WG tunnels are unusable for few seconds...

This is the log from the backup FW during save of HA settings:

Code Select Expand

<13>1 2026-03-10T07:47:28+00:00 fw2.internal kernel - - [meta sequenceId="24"] <6>[3281] carp: 10@vtnet1: BACKUP -> MASTER (preempting a slower master)
<13>1 2026-03-10T07:47:28+00:00 fw2.internal kernel - - [meta sequenceId="25"] <6>[3281] carp: 10@vtnet0: BACKUP -> MASTER (preempting a slower master)
<13>1 2026-03-10T07:47:28+00:00 fw2.internal kernel - - [meta sequenceId="26"] <6>[3281] arp: 192.168.1.1 moved from 00:00:5e:00:01:0a to 52:54:00:40:0c:64 on vtnet1
<13>1 2026-03-10T08:47:28+01:00 fw2.internal opnsense 5030 - [meta sequenceId="27"] /usr/local/etc/rc.syshook.d/carp/20-openvpn: Carp cluster member " (192.168.1.1) (10@vtnet1)" has resumed the state "MASTER" for vhid 10
<13>1 2026-03-10T08:47:28+01:00 fw2.internal opnsense 5999 - [meta sequenceId="28"] /usr/local/sbin/pluginctl: plugins_configure crl (1)
<13>1 2026-03-10T08:47:28+01:00 fw2.internal opnsense 5999 - [meta sequenceId="29"] /usr/local/sbin/pluginctl: plugins_configure crl (execute task : core_trust_crl(1))
<13>1 2026-03-10T08:47:28+01:00 fw2.internal opnsense 13189 - [meta sequenceId="30"] /usr/local/etc/rc.syshook.d/carp/20-openvpn: Carp cluster member " (192.168.122.10) (10@vtnet0)" has resumed the state "MASTER" for vhid 10
<13>1 2026-03-10T08:47:28+01:00 fw2.internal opnsense 5999 - [meta sequenceId="31"] /usr/local/sbin/pluginctl: plugins_configure crl (execute task : openvpn_refresh_crls(1))
<13>1 2026-03-10T08:47:28+01:00 fw2.internal opnsense 16795 - [meta sequenceId="32"] /usr/local/sbin/pluginctl: plugins_configure crl (1)
<13>1 2026-03-10T08:47:28+01:00 fw2.internal opnsense 16795 - [meta sequenceId="33"] /usr/local/sbin/pluginctl: plugins_configure crl (execute task : core_trust_crl(1))
<13>1 2026-03-10T07:47:28+00:00 fw2.internal kernel - - [meta sequenceId="34"] <6>[3281] carp: 10@vtnet0: MASTER -> BACKUP (more frequent advertisement received)
<13>1 2026-03-10T07:47:28+00:00 fw2.internal kernel - - [meta sequenceId="35"] <6>[3281] wg0: link state changed to UP
<13>1 2026-03-10T08:47:28+01:00 fw2.internal opnsense 20179 - [meta sequenceId="36"] /usr/local/etc/rc.syshook.d/carp/20-openvpn: Carp cluster member " (192.168.122.10) (10@vtnet0)" has resumed the state "BACKUP" for vhid 10
<13>1 2026-03-10T08:47:28+01:00 fw2.internal opnsense 16795 - [meta sequenceId="37"] /usr/local/sbin/pluginctl: plugins_configure crl (execute task : openvpn_refresh_crls(1))
<13>1 2026-03-10T08:47:28+01:00 fw2.internal opnsense 24338 - [meta sequenceId="38"] /usr/local/sbin/pluginctl: plugins_configure crl (1)
<13>1 2026-03-10T08:47:28+01:00 fw2.internal opnsense 24338 - [meta sequenceId="39"] /usr/local/sbin/pluginctl: plugins_configure crl (execute task : core_trust_crl(1))
<13>1 2026-03-10T08:47:28+01:00 fw2.internal opnsense 24338 - [meta sequenceId="40"] /usr/local/sbin/pluginctl: plugins_configure crl (execute task : openvpn_refresh_crls(1))
<13>1 2026-03-10T08:47:29+01:00 fw2.internal opnsense 30806 - [meta sequenceId="41"] /usr/local/etc/rc.syshook.d/carp/20-openvpn: Carp cluster member " (192.168.1.1) (10@vtnet1)" has resumed the state "BACKUP" for vhid 10
<13>1 2026-03-10T08:47:29+01:00 fw2.internal opnsense 33152 - [meta sequenceId="42"] /usr/local/sbin/pluginctl: plugins_configure crl (1)
<13>1 2026-03-10T08:47:29+01:00 fw2.internal opnsense 33152 - [meta sequenceId="43"] /usr/local/sbin/pluginctl: plugins_configure crl (execute task : core_trust_crl(1))
<13>1 2026-03-10T07:47:29+00:00 fw2.internal kernel - - [meta sequenceId="44"] <6>[3282] carp: 10@vtnet1: MASTER -> BACKUP (more frequent advertisement received)
<13>1 2026-03-10T08:47:29+01:00 fw2.internal opnsense 33152 - [meta sequenceId="45"] /usr/local/sbin/pluginctl: plugins_configure crl (execute task : openvpn_refresh_crls(1))
<13>1 2026-03-10T07:47:29+00:00 fw2.internal kernel - - [meta sequenceId="46"] <6>[3282] wg0: link state changed to DOWN