OPNsense Forum
English Forums => High availability => Topic started by: Grossartig on May 11, 2022, 04:17:10 am
-
My internet provider (Verizon FiOS) is only giving me a single WAN IP, so I was looking for a way to implement full HA within that limitation. From all my reading, I understood that the traditional CARP tutorials (https://docs.opnsense.org/manual/how-tos/carp.html (https://docs.opnsense.org/manual/how-tos/carp.html) and https://www.thomas-krenn.com/en/wiki/OPNsense_HA_Cluster_configuration (https://www.thomas-krenn.com/en/wiki/OPNsense_HA_Cluster_configuration)) require one to have three static IP addresses on the WAN -- one for each OPNsense box and another one for the VIP that ties both of them together.
I then came across this article (https://www.thegeekpub.com/5688/ethernet-switch-between-the-ont-and-the-router-fios/ (https://www.thegeekpub.com/5688/ethernet-switch-between-the-ont-and-the-router-fios/)) that describes a scenario (option 2) where a managed switch can be put between the ONT and the two OPNsense boxes, and an untagged VLAN needs to be created across three physical Ethernet ports on that switch -- one for each OPNsense box and another for the cable that goes into the Ethernet port of the ONT.
On the surface, I didn't understand how that should work, but somehow it does, provided that both WAN ports of the OPNsense boxes share the same MAC address.
Somehow can perhaps educate me what the managed switch does that:
1) Enables it to handle two connected devices that have the same MAC address
2) Pretends towards the ONT that only a single device is attached
Both OPNsense boxes show an active WAN connection with the same externally assigned dynamic IP address. I've run several tests to confirm that connectivity is positively there, both through Master and Backup. I also shut down either one and the other keeps on working as it should in a proper HA configuration.
The switch I bought is the D-Link DGS-1100-08V2. I configured it to have a dedicated management port so that it can only be administered from a directly attached Ethernet cable.
Note: I do have CARP/pfSync/XMLRPC properly set up on the LAN, but not at all on the WAN side. The latter appears to be fully taken care off by the managed switch.
-
This is how I built HA - you should always plug external connections into a proper switch first for monitoring und debugging purposes...
https://forum.opnsense.org/index.php?topic=18732.msg85748#msg85748
-
Interesting -- you even have two switches on the WAN side of your setup :)
I'm quite happy with the switch-on-WAN side solution, but i'm still at a loss as to how it actually works. Both OPNsense boxes on my end share the same WAN MAC (on purpose, to make failover easier, as my provider doesn't like it when a different device suddenly connects to the ONT). But how does the switch not get confused by two identical MACs being connected to it?
I'm not complaining, it just works (and works well), but I don't quite understand why :)
-
The Switches are stacked and act as one device. If one of them fails/reboots the remaining one takes over - no LAGG is in use.
The connection to the provider is VRRP based. They have two routers siiting in the same LAN with dedicated IPs and like on the OPNsense only one device is answering requests. So there are no mutiple MAC addresses...