[Solved] CARP Setup - xml file ?

[remd]

I've followed this guide to setup CARP, but I havent been able to set it up successfully so far - https://docs.opnsense.org/manual/how-tos/carp.html

There is one catch there are two interfaces I should map differently, the hardware is pretty much the same except that one firewall has two additional SPF+ ports, so I need to map port 7,8 to 2,3, but I've been told thats doable by editing the xml file.
The problem is that I don't see anywhere in the documentation at what point and where you can export/import that xml file ?
I went through the steps of creating the same interfaces on both firewalls (only diff is wan/lan on port 2,3 instead of 7,8), setup the firewall rules to accept CARP packages, setup the CARP interfaces (they can ping eachother), setup the virtual ips, updates the outbound NAT, then no dhcp on these two firewalls, and then started to setup the HA sync part, but at this point I guess it couldnt work as I had to edit the xml file to have the correct mapping and I don't know where/how to do this ?
I tried save anyway to see if there may have been some clever abstraction of the underlying port and that the interface name may have been enough, but that didnt work - backup server not reachable.

I may be missing something obvious, but it would be helpful if someone could point me in the right direction.

fw CARP logs after hitting save:
"
Sep 5 21:53:17   mainfw kernel: carp: demoted by -240 to 0 (pfsync bulk done)
Sep 5 21:53:17   mainfw kernel: carp: demoted by 240 to 240 (pfsync bulk start)
Sep 5 21:52:38   mainfw kernel: carp: demoted by -240 to 0 (pfsync bulk fail)
Sep 5 21:51:28   mainfw kernel: carp: demoted by 240 to 240 (pfsync bulk start)
"
using the latest 17.7.1 production version.

[remd]

It seems to have synchronised at least in part... (I was missing a configuration option), I'll check the interfaces, but I doubt it synched them, so the question about the xml file probably remains..

[remd]

Its a bit strange about 3/4 of the configuration has been transferred to the other firewall by CARP but not all, maybe it takes some more time.. ?
And now its saying again that the backup firewall is not available after it was showing that status and I refreshed..

It also showed one error on the backup fw - 09-05-17 22:18:55 [ Sorry but we could not find a required assigned ip address on the interface for the virtual IP address 192.168.x.x.]
But that is only one of 15 interfaces.

[remd]

don't know if the sync broke anything, but now the first CARP interface cannot ping the second one, whereas the second one can ping the first one, so maybe some firewall rule.
It might make sense as it seems to have copied over about half of the rules and stopped, maybe some rule works on the first firewall but broke something on the second one  ?

Also the packet capture shows icmp packets sent and received, but they are not showing in the interfaces, diag, ping interface.
UPDATE: somehow the CARP GW settings got messed up, it seems to work fine after fixing it, but this may be a problem anyway if it messes it up after it syncs..

anyway still investigating, but it would still be helpful to have some insight from anyone having configured this already and the steps I didnt find like how to access that xml file .. 

[remd]

Ok, I've reproduced the error, it is indeed related to the 2 ports that are different from one to the other fw.
So.... I need to be able to edit the HA xml file, which I haven't found yet to map the ports.

So....could anyone let me know how to export/import that file ??

Thx!

[remd]

I guess the xml file is available in a partial config backup, I'll give that a try..

[remd]

ok, so I've downloaded the backup xml conf file, then edited the igb2 and 3 and replaced them with ix0 and ix1 , which are the two interfaces on the main firewall I want to map to the 2nd and 3rd port of the second firewall, and then restored the configuration, but it apparently only updates the VLAN's and not the interfaces, and the interfaces are set to em0 afterwards and ix0 and 1 are not available in the list, so I guess this suggestion is not working, unless there is something else that should be done ?

I also tried to rename the interfaces in command line as such:
in /etc/rc.conf (to reassign at boot)
ifconfig_igb2_name="ix0"
ifconfig_igb3_name="ix1"
ifconfig_igb3_vlan10_name="ix1_vlan10"
ifconfig_igb3_vlan20_name="ix1_vlan20"
ifconfig_igb3_vlan30_name="ix1_vlan30"
ifconfig_igb3_vlan50_name="ix1_vlan50"

and ifconfig igb2 name ix0 etc...

Then in the gui igb2 and 3 were mapped to em0 again, but I could set them to ix0 and 1 this time.
So it seemed ok, but after a reboot the vlan's were still mapped to ix1_ but the interfaces were back to em0, so somehow it is not saving the interfaces name.

Anyone knows how to change the interface name permanently, or any suggestion to make this work ?

Thanks !

[remd]

I also tried this in /etc/rc.conf:
in /etc/rc.conf
ifconfig_igb2_name="ix0"
ifconfig_ix0="inet 192.168.x.x netmask 255.255.255.248"
ifconfig_igb3_name="ix1"
ifconfig_igb3_vlan10_name="ix1_vlan10"
ifconfig_ix1_vlan10="inet 192.168.x.x netmask 255.255.254.0"
ifconfig_igb3_vlan20_name="ix1_vlan20"
ifconfig_ix1_vlan20="inet 192.168.x.x netmask 255.255.254.0"
ifconfig_igb3_vlan30_name="ix1_vlan30"
ifconfig_ix1_vlan30="inet 192.168.x.x netmask 255.255.254.0"
ifconfig_igb3_vlan50_name="ix1_vlan50"
ifconfig_ix1_vlan50="inet 192.168.x.x netmask 255.255.255.0"
"
But that didnt help, somehow only the vlans are sticking correctly after a reboot.

In any case even if I didnt reboot, and all used interfaces matched on both firewalls, it still mixed them up in some places, like the virtual ips weren't on the right interface and two vlan firewall rules didnt update, while all others did.
I could fix this manually, but that doesnt seem like a stable solution, and then there is the issue when the firewall reboots also.

I'm not sure what to try next at this point, any suggestions are welcome

[remd]

Regarding keeping the interface configuration after a reboot, I've tried to set the configuration in /etc/rc.conf , /etc/rc.conf.local , /etc/rc.conf.d/ and in /usr/local/etc/rc.d/ but it seems to be ignored at startup.

The ix interfaces go to em0 and the igb2/3 interfaces appear again but not set.
so it seems to default to em0 when it doesnt know what the interface is and it insist on making igb2/3 available.

There are network related scripts in rc.bootup, but this is all php and calls other files so not sure if it has an influence and how it could be customized. Then maybe there is the /usr/local/etc/rc file...

In any case still searching, seems like I'll spending some sleepless nights on this one  .. :/

[remd]

Thanks to a tip from the deciso support, I've done it the other way around and it worked.

The trick was to get the xml backup file from the main firewall, to
modify the 2 fiber interfaces names to the corresponding interfaces on
the other firewall, and not to modify anything else.
Then upload the configuration on the secondary firewall, not reboot and
modify all the remaining settings to correspond to that firewall (ips,
hostnames, etc..), then the synchronisation works, and survives a reboot.
Sounds simple enough when you have the right procedure