debugging pfsync

[toxic]

Hello,

a long time ago I got states to be synchronized between my 2 firewalls both running latest opnsense and similar hardware.

They are both running on proxmox with a virtual NIC that is a linux bond on the host, WAN has no VLAN but I have several VLANs for LAN, the bond on proxmox trunks them all and I devined all the vlans on the virtual interface in opnsense.

All seems to work well, again I had it working a while ago and can't find what I did to break it...

I have CARP failover that works (althought all sessions gets killed since no states are synced)
I have even recently tried to add the pfsync0 to the carp group (no idea what it does...)
:

Code: [Select]

main$ifconfig pfsync0
pfsync0: flags=41<UP,RUNNING> metric 0 mtu 9000
        pfsync: syncdev: vtnet1_vlan9 syncpeer: 10.0.9.3 maxupd: 128 defer: off
        syncok: 1
        groups: pfsync carp


Code: [Select]

backup$ifconfig pfsync0
pfsync0: flags=41<UP,RUNNING> metric 0 mtu 9000
        pfsync: syncdev: vtnet1_vlan9 syncpeer: 10.0.9.2 maxupd: 128 defer: off
        syncok: 1
        groups: pfsync carp

I have all interfaces in the same order, the same underlying name...

on both I setup the Synchronize Peer IP properly and they both use the same syncdev as you can see.
But when I do a tcpdump on this interface, vtnet1_vlan9 interface, I only see CARP traffix (heartbeats or what they are called in CARP world...)
I see nothing on UDP or PFSYNC protocol that would share states.

In the GUI, I see something strange in interface overview, see the attachment below. I get the same on both frewalls, only errors (and different number of errors but hey...)


If someone has any clue as to a way to debug pfsync0, see what are these erros, maybe get some logs...

I wanted to crast a "fake" pfsync packet and send it out trough vtnet1_vlan9 just to make sure it's not being blocked by some unknown rule that isn't logging, but I don't see many packets being blocked, and I do have a rule on the proper interface to allow any to any on protocol PFSYNC...

Any help in investigating is realy welcome !