Gateway to master instance makes backup instance unreachable on LAN

[raspbeguy]

Hello,
I'm pretty new with OPNsense/freeBSD. I have to OPNsense instances in master/backup setup.

• Instance mulder, LAN IP 192.168.0.4/24
• Instance scully, LAN IP 192.168.0.5/24

Both instances LAN interfaces are configured in a CARP: 192.168.0.1/24

Currently mulder is backup and scully is master.
To enable mulder able to access WAN, I set up a Gateway in System > Gateways

Then I set this gateway as IPv4 Upstream Gateway in Interfaces > [LAN]


As a result, the backup instance has access to the internet but is now unreachable from my LAN devices: whenever it receives a TCP or ICMP package, it wants to respond through the configured gateway. To make those screen capture I had to connect through another interface.

Am I missing something?

[Patrick M. Hausen]

You need a fixed IP address on WAN for each of the notes both pointing to the regular default gateway, and a floating (CARP/VIP) IP address on WAN that is active on whichever node is the master.

This way both nodes have Internet access.

[raspbeguy]

Well, this isn't possible here. WAN side is managed by DHCP so only one possible IP. There is already a topic for that but that's another problem

[Patrick M. Hausen]

Well, what I told you is how the setup and protocol is supposed to work. If you only have a single uplink with a single IP address via DHCP, what's the point having a HA setup, anyway?

Of course you can try to hack until it sort of works. Unfortunately I cannot help with that.

[raspbeguy]

Of course, it would make more sense to have two separate physical links to connect to the WAN. But it still makes sense to set up HA even in this configuration, for example when upgrading the instances without downtime, and of course educational purpose and having fun (maybe I should have made it clear that this isn't a production setup, only my homelab).