Hi Team,
I'm recently installed opnsense in VMware ESXI platform where deployment completed and try to make the CARP between the two machine gets formed but unfortunately I'm receiving master on both machine.
I did TS on Rules where placed correctly and HA SYNC working as expected and I could see the CARP protocal running (224.0.0.1 and 224.0.0.18) in interfaces.
I tried restart, carp disable, persistence mode those are not helping and thoroughly checked that Virtual IP configuration where placed as it is.
I do have receiving IANS arp from connected top layer physical firewall
100.100.102.250 7 00:00:5e:00:01:0a >>>> VIP
100.100.102.252 0 00:50:56:90:01:6a >>>> FW1
100.100.102.253 1 00:50:56:90:a5:82 >>>> FW2
I did upgrade the system and still issue remain the same.
CARP issue faced version of OPNsense 24.1 and 24.1.5_3-amd64
General Log file that we receiving both machines,
2024-04-17T23:46:57 Notice opnsense /usr/local/etc/rc.syshook.d/carp/20-openvpn: Resyncing OpenVPN instances for interface (100.100.102.250).
2024-04-17T23:46:57 Notice opnsense /usr/local/etc/rc.syshook.d/carp/20-openvpn: Carp cluster member " (100.100.102.250) (10@vmx0)" has resumed the state "MASTER" for vhid 10
2024-04-17T23:46:53 Notice opnsense /usr/local/etc/rc.syshook.d/carp/20-openvpn: Resyncing OpenVPN instances for interface (100.100.102.250).
2024-04-17T23:46:53 Notice opnsense /usr/local/etc/rc.syshook.d/carp/20-openvpn: Carp cluster member " (100.100.102.250) (10@vmx0)" has resumed the state "BACKUP" for vhid 10
2024-04-17T17:47:42 Notice opnsense /usr/local/etc/rc.syshook.d/carp/20-openvpn: Resyncing OpenVPN instances for interface (100.100.102.250).
2024-04-17T17:47:42 Notice opnsense /usr/local/etc/rc.syshook.d/carp/20-openvpn: Carp cluster member " (100.100.102.250) (10@vmx0)" has resumed the state "MASTER" for vhid 10
2024-04-17T17:47:38 Notice opnsense /usr/local/etc/rc.syshook.d/carp/20-openvpn: Resyncing OpenVPN instances for interface (100.100.102.250).
2024-04-17T17:47:38 Notice opnsense /usr/local/etc/rc.syshook.d/carp/20-openvpn: Carp cluster member " (100.100.102.250) (10@vmx0)" has resumed the state "BACKUP" for vhid 10
So, can anyone help me out to how to resolve this issue ?
Thanks,
Athisesan R
Since CARP sends broadcasts with the VRRP protocol with forged MAC addresses, you need to allow that in your ESXI vswitch.
Hi Monviech,
I'm using dvs with portgroup, so is it required to enable forged transmits from dvs port-group security ?
Thanks,
Athisesan R
Sorry I'm not sure on that one. I just know that there cant be mac security enabled, and maybe even promiscious mode is needed. But a little unsure.
Quote from: athisesanr on April 17, 2024, 09:18:57 PM
Hi Monviech,
I'm using dvs with portgroup, so is it required to enable forged transmits from dvs port-group security ?
Thanks,
Athisesan R
Should work, yes, there is a guide in pfsense docs (which is not related to *sense)
Hi
I did promiscuous mode enable on connected dvs port group and observing status as both firewall is "Backup" now.
I couldn't get it where the pfsense or vmware vrrp docs where works for here.
https://communities.vmware.com/t5/vSphere-vNetwork-Discussions/Can-t-ping-virtual-router-IP-in-VRRP/td-p/854331
https://www.reddit.com/r/vmware/comments/hh63yd/dvswitch_not_passing_multicast/
Thanks,
Athisesan
Hi Team,
Finally, I fixed CARP issue on vmware esxi dvs level with using vmware mac learning option.
follow the steps.
- Form the VIP between the Opnsense FWs
- Edit the DVs port group security from vcenter
- setting changes likes
Promiscuous mode - Reject
MAC address changes - Reject
Forged transmits - Reject
MAC Learning
Status - Enabled
Allow unicast floodin - Enabled
MAC limit - 4096
MAC limit policy - Allow
https://docs.vmware.com/en/VMware-vSphere/8.0/vsphere-networking/GUID-E0246B3D-9FB1-4976-8217-5C085863EA9A.html
Thanks,
Athisesan R