Messages

I have confirmation that static routing unfortunately is not supported, which might explain the issues since that is what I was trying. Is there any solution in this situation in which I can realise a highly available setup perchance?

I got a reply from the ISP. Indeed the range is not statically routed. the nearest switch has a .1 configured as gateway. So if I ARP that IP I will receive the traffic.

I suspect that for a HA setup I have to create many IP Aliases, or does ProxyARP also work?

Hey, it appears that the first IP is in use by the ISP, so I took the second as my gateway with an IP Alias. If I do not proxy ARP then the ISP is unaware of how to route my VM since I can see it searching for the VM's IP via ARP. Proxy ARP fixes this for me, but I feel that proxy ARP is not a proper solution.

Any suggestions on how to 'dodge' proxy ARP are welcome.

WAN is indeed in a different /29 subnet which is mostly already used with static IP's. I suspect it is as described in the PFsense documentation as transport or interconnect network. My secondary network with public IP's is a /24 from an entirely different range.

Assign the first address of that subnet to an interface of OPNsense, e.g. OPT1. Connect VMs to that interface, give VMs the remaining IP addresses with the OPNsense address as the default gateway. Disable NAT.

While RFC 1918 did introduce a concept of "public" and "private" IP addresses, nowhere does it say, that public ones must be placed on a "WAN" interface. And proxy ARP is evil.

Just configure your interfaces and let routing do its magic.

Heey, thanks for your reply! If possible I could use another small suggestion  ::)  I tried this setup this morning and it works, but only with ProxyARP. I have a /24 currently. Preferably I would like to dodge NAT.
- .1 is routed to the ISP's gateway
- .2 is my IP Alias on the interface
- .3 is my VM

I am getting the feeling that there is no static route to my .2, given that I can see .1 desperately ARPing for .3 when I have no proxy ARP. I am also

Now it is my intention to run this in HA mode, is that possible with Proxy ARP our should I add 253 addresses as IP alias individually?

Thanks again for thinking with me on this  :)

Thanks for the suggestion, that does seem to work  :) Unfortunately some apps PPTP/Wireguard/IPSec do benefit from having the external IP assigned to the interface. I use this to remain connected, but also to P2P to other networks. Do you perchance see any solution which allows me to use external IP's directly on VM's?

Thanks for your reply. I suspect the latter, that they provided me a subnet. When I ping one of the IP's in the subnet from a VPS I can see it arriving on my WAN interface which is Proxy ARPing the entire range. Since it is an residential connection this is not within their normal flow, so I have little instructions to go on nor do I expect much support from their side.

But I can confirm that traffic able to go out from the subnet and is also being routed back properly. Just getting it back to the sender is challenging.

Hi all,

I have a somewhat strange setup that I would like to validate. After a few hours of trying to get it to work I started wondering if it was possible at all.

My ISP issued me a subnet on my request to use for my homelab to give VM's external IP's without NAT.

- My firewall (DEC3862) is hooked directly to the ISP.
- Proxy ARP has been setup and appears to work
- Outbound NAT is excluded for the public IP range
- When I put the VM with the public IP behind the LAN interface. I get the following result, which I understand since the IP is not in the subnet of the interface.

Code Select Expand


┌────────────┐         ┌─────────────┐           ┌────────────┐
│     VM     ├────────►│    Router   ├──────────►│  Internet  │
│  1.2.3.4   │         │  10.234.0.1 │           │   8.8.8.8  │
│            │     x───┼             ◄───────────┤            │
└────────────┘         └─────────────┘           └────────────┘


When I create a VLAN without IP and point it directly at the external gateway the VM keeps ARPing for that gateway which I understand.

After a lengthy search I understand that bridge mode/transparant mode or 1-to-1 NAT should work. The first one is a bit much since I also have a household that requires regular LAN services. The 1-to-1 NAT or port-forwarding would solve the problem, but makes learning virtualised routers/firewalls a lot harder since I'd have to remap everything.

So my question, given that the traffic is almost going full circle. Is there any way to convince the router to accept the ARP entry for my VM for an IP that is not in it's subnet and route in that direction without a full WAN/LAN bridge. Or is there perhaps any other way to do this? Thanks!

Problem turns out to be related to the Connect-X4 driver in BSD. I picked up some DEC3800's and it worked instantly with the same config. Amazing devices, highly recommended if you want something that just works.

Resolved (kinda)

[Master (in 'Persistent maintenance mode')]

Hi there,

For my lab setup I am looking to use CARP and outbound NAT. Unfortunately I can't get it to work consistently. I am looking for suggestions as debugging approach, or someone with similar experience  ;)

I've tried to swap out the switches on the WAN side (Unifi to Mikrotik) and go to a different subnet. The uplink is a Connect-X4 since I am planning on upgrading the LAB to 10G.

LAN VIP switching is working fine, different network adapter though (igbe). As soon as I enter maintenance mode it swaps to backup and the standby assumes the master.


Master (in 'Persistent maintenance mode')
2024-05-06T16:41:41   Notice   kernel   <6>carp: 6@mce0: MASTER -> BACKUP (more frequent advertisement received)   
2024-05-06T16:41:41   Notice   kernel   <6>carp: 6@mce0: BACKUP -> MASTER (master timed out)

Standby (that should become master):
2024-05-06T16:41:41   Notice   kernel   <6>carp: 6@mce0: MASTER -> BACKUP (more frequent advertisement received)   
2024-05-06T16:41:41   Notice   kernel   <6>carp: 6@mce0: BACKUP -> MASTER (preempting a slower master)

I've checked the package dump and can see CARP traffic on both sides. Not sure about the interval though.

Any and all suggestions are more then welcome.