Config Sync (XMLRPC) not Syncing to Secondary

[kbrown.it]

I am setting up OPNSense (Version: 22.7-amd64, FreeBSD 13.1-RELEASE, OpenSSL 1.1.1q 5 Jul 2022) to be in an HA Cluster with another OPNSense firewall (both were installed at the same time with the same installer, I use the Serial Console Installer). I have them configured and CARP setup (which is working), but when I went to set up the HA Config Sync I cannot get the config to sync over.

When I try and preform a Sync, I see it does the "pfsync bulk start" and thank it does a "pfsync bulk done" shortly after on the console. When I got to Status all I see is "The backup firewall is not accessible or not configured.". I have compared the settings to another OPNSense cluster (older version) and they are similar (different IPs and and different options selected on what to sync). I turned on logging for the rule on my HA interface (which is wide open) and I can see the traffic being allowed. I do not see anything in the logs specific to the syncing.

I am at a loss as to why the Config Sync is not working as it should (or if it is working and the Status page is broken). I have seen post were people talked about a semi-colon in the password being an issue. I am not using a semi-colon in the password and after removing all special characters, the issue was still present (so I do not thing that is the issue). The Web Interface is allowed on all interfaces as well.

[Patrick M. Hausen]

Config of the HA interfaces on both nodes, HA config on the master, firewall rules for that interface on both nodes, please.

[kbrown.it]

HA Config and Rules are configured on both sides (forgot to mention this in the original post).

[Patrick M. Hausen]

Please post the configuration of the HA interfaces of both sides, the HA config of the master node, and the firewall rule(s) applied to the HA interfaces of both sides.

Complete screenshots with all settings.

Without that it is impossible to help you. HA and config sync works. We need to find out what is wrong eith your particular setup. To do that we need at least the things I asked for in my last post, already.

"I configured it" is not helpful in diagnosing a problem. If you configured it correctly, it would work as designed, wouldn't it?

[kbrown.it]

If you configured it correctly, it would work as designed, wouldn't it?

Assuming nothing has broken within the last code up date. As mentioned, I have this working correctly (and mirrored the setup to this new one) on OPNsense 21.7.3_3-amd64 in a VM Environment. Comparing the two HA Settings pages, I see my newer one has an additional option (Disconnect dialup interfaces) and the Synchronize States is not at the top (which it is in the older version). This indicates that there have been some changes to at least the look/function of the page (which can result in breaking functionality if stuff is not coded right).

I will post the requested screenshots shortly.

[kbrown.it]

Due to the restriction of 4 attachments per post, here is the Primary Side HA. Primary HA shows the top part of the HA configuration. Below that are the check boxes as to what to sync (I did not screenshot all of them, but below is what is checked). Primary HA Int 1 and 2 are the Interface for the HA (this is a physical port with a cable connected directly to the other firewall's HA port). Primary HA Rule shows the rules setup for the HA port (the only Floating rules are the "Automatically generated rules" when you install OPNsense).

I have also done a ping test and the Primary can ping the Secondary's IP Address and vice versa.

Configuration Synchronization Settings (list subject to change as needs change):
Dashboard
Users and Groups
Auth Servers
Virtual IPs
Static Routes
Network Time
Firewall Groups
Firewall Rules
Firewall Schedules
Firewall Categories
Firewall Log Templates
Unbound DNS

[kbrown.it]

Due to the restriction of 4 attachments per post, here is the Secondary Side HA. Secondary HA shows the top part of the HA configuration. Below that are the check boxes as to what to sync (I did not screenshot all of them, but below is what is checked). Secondary HA Int 1 and 2 are the Interface for the HA (this is a physical port with a cable connected directly to the other firewall's HA port). Secondary HA Rule shows the rules setup for the HA port (the only Floating rules are the "Automatically generated rules" when you install OPNsense).

I have also done a ping test and the Secondary can ping the Primary's IP Address and vice versa.

Configuration Synchronization Settings (list subject to change as needs change):
Dashboard
Users and Groups
Auth Servers
Virtual IPs
Static Routes
Network Time
Firewall Groups
Firewall Rules
Firewall Schedules
Firewall Categories
Firewall Log Templates
Unbound DNS

[kbrown.it]

I did notice I did not post the hardware (in case that will be a factor). I am using two Protectli FW6A units. They both have 120 SSD, 8 GB of RAM, a Intel Celeron 3867U @ 1.80 GHz (2 core, 2 thread), and the Network adapters are Intel I211 (identified as igb). The SSD was setup with zfs using the Console Install Package.

[arcad]

Hello,
I have the same problem, with the difference that the HA system worked for 2 years with all various upgrades, but for about 1 month it no longer works, after the upgrade to version 22.7
I also tried to swap the master with slave but I get the same result.
Between the various tests I restored the slave to factory settings, I reconfigured the interfaces to adapt them to work with the CARP and the first synchronization took place without problems. The slave took all the rules, users, certificates, etc ... but the HA function stopped working soon after.
If I manually enable the "Persistent CARP maintenance mode" the slave starts working quietly. The only thing that doesn't work is the synchronization of the rules between the two firewalls.
My version is 22.7.1
Hardware i7-4770 CPU @ 3.40GHz (4 cores, 8 threads)

[kbrown.it]

I will note, that yes it does say "Do not configure XMLRPC sync on the backup firewall" in the document. I will say the VM Firewall we have setup (OPNsense 21.7.3_3-amd64), which was setup by a vendor who uses OPNsense for some of their stuff, has it setup with XMLRPC on both the Primary and Backup firewalls and that works without issue. The sync mainly only happens from the Primary to the Backup, but in the event the Backup becomes the Primary (due to hardware failure of the Primary) I will need to have the Backup Sync to the new Primary if anything changes during that time.

Removing the config for XMLRPC on the Backup has not solve the issue either.

[ofu12345]

Hi,
maybe I am experiencing something similar here.
Both my primary and my backup firewalls are VMs (on Proxmox VE in this case). So this means, the synchronize Interface is virtualized.

When XMLRPC Sync is configured to sync "Firewall Rules" and I trigger an update from primary to secondary, the firewall rules on the destination disappear.


How to fix:
- do not sync firewall rules
- sync config, manually add rules on synchronize interface, restart broken transfer


My wild guess:
During synchronisation all firewall rules are deleted and then recreated. But when the rules are deleted, adding new ones is not possible any more.

both systems are running 22.7.5, everything else works as expected.
What else can I provide to solve this issue?

System - High Availability - Settings:
Disable Preempt: yes
Synchronize States: yes
Synchronize Interface: firewall (SRC system uses 192.168.250.10/24)
Synchronize Peer IP: 192.168.250.11
Syncronize Config to IP: 192.168.250.11
Remote System Username: root
Remote System Password: supersecret123
Things to sync: Dashboard, Virtual IPs, Static Routes, WebGUI, NAT.



I disabled CARP completely to better find out what is happening here.

Thanks,
 Oliver

[ofu12345]

I found my configuration error:
Interfaces were not setup in exactly the same order on both systems.

Sorry for bothering,
 Oliver