Outbound NAT Problem

[vertices]

I'm having a problem with an Azure hosted OPNsense 23.1.9 deployment. VM's are SNATing to the wrong IP.

In Azure, I have a 4 IP address prefix, so 4 contiguous addresses. The first is assigned to the VM interface, and the next 3 are added to the same VM interface as additional IP configurations. As an example, in Azure:

ipconfig1 - Primary - Private IP 10.40.10.1 - 20.101.101.101
ipconfig2 - Secondary - Private IP 10.40.10.2 - 20.101.101.102
ipconfig3 - Secondary - Private IP 10.40.10.3 - 20.101.101.103
ipconfig4 - Secondary - Private IP 10.40.10.4 - 20.101.101.104

In my OPNsense config, it can't see the public IPs due to Azure, only the private listed above. So ipconfig1 is assigned to the WAN interface. Then there are VIPs created for next 3 addresses.

WAN - 10.40.10.1
VIP1 - 10.40.10.2
VIP2 - 10.40.10.3
VIP3 - 10.40.10.4

For some reason, even though all outbound NAT configs are set to use WAN address, they don't, they SNAT to VIP1. Now if I change the outbound NAT rule to VIP2 or VIP3, they will properly SNAT to the set VIP. But for some reason, they just choose VIP1 when I set it to WAN. They should SNAT out the WAN address.

I should add that all three of the VIPs are set to "Deny Service Binding".

I haven't been able to track down why this is. Any ideas why VMs would SNAT using a VIP rather than the WAN address when they are set to use WAN?


TIA!

[zan]

If you set Interface address as 'translation/target' addres of your SNAT rules they usually have this format (You can check with "pfctl -sn" command):
     "nat on interface from source to any -> (interface:0)"

(interface:0) will be translated to the first address of interface.

Since your WAN interface have multiple addresses, my best guess is your VIP1 somehow end up got picked by system as the first address.
You may want to set the 'translation/target' address of your SNAT rule to 10.40.10.1 explicitly as a workaround.

[vertices]

I'll check with pfctl as soon as I get a chance. I did try setting my NAT rule explicitly to the IP 10.40.10.1 but at that point VMs in that spoke using that rule couldn't get out at all. So not sure what that's all about.

[vertices]

no nat proto carp all
nat on hn0 inet from 127.0.0.0/8 to any port = isakmp -> (hn0:0) static-port
nat on hn0 inet from 127.0.0.0/8 to any -> (hn0:0) port 1024:65535
nat on hn0 inet from <WindowsVMSubnet> to any -> (hn0:0) port 1024:65535
nat on hn0 inet from <hub1_snet1> to any -> (hn0:0) port 1024:65535
nat on hn0 inet from <hub1_snet2> to any -> (hn0:0) port 1024:65535
nat on hn0 inet from <spoke1_snet1> to any -> (hn0:0) port 1024:65535
nat on hn0 inet from <spoke2_snet1> to any -> (hn0:0) port 1024:65535
nat on hn0 inet from <spoke3_snet1> to any -> (hn0:0) port 1024:65535
nat on hn0 inet from (hn1:network) to any port = isakmp -> (hn0:0) static-port
nat on hn0 inet from (lo0:network) to any port = isakmp -> (hn0:0) static-port
nat on hn0 inet from 127.0.0.0/8 to any port = isakmp -> (hn0:0) static-port
nat on hn0 inet from (hn1:network) to any -> (hn0:0) port 1024:65535
nat on hn0 inet from (lo0:network) to any -> (hn0:0) port 1024:65535
nat on hn0 inet from 127.0.0.0/8 to any -> (hn0:0) port 1024:65535
nat-anchor "acme-client/*" all
no rdr proto carp all
rdr-anchor "acme-client/*" all

That's the output. In the above example, I am testing with VMs in hub1_snet1.

nat on hn0 inet from <hub1_snet1> to any -> (hn0:0) port 1024:65535

That rule is what I have been testing with, changing from WAN to VIP2 or VIP3, and it does change SNAT to VIP2 or VIP3 when I do that. But when I leave it at default WAN it uses VIP1.

[Patrick M. Hausen]

I am not questioning that your observed behaviour is if not a bug at least a severe POLA violation.

Nonetheless with my own OPNsense systems I prefer to set addresses explicitly and use aliases for "inline documentation".

OPNsense does not let me group aliases according to type so I have come to use a particular naming scheme. Imagine you have three customer servers A, B, C. I then create these objects:

Host4_Customer_A_Ext
Host4_Customer_A_Int
Host4_Customer_B_Ext
Host4_Customer_B_Int
Host4_Customer_C_Ext
Host4_Customer_C_Int

with - I hope - obvious meanings. And I use these exclusively in rules. I always use aliases in rules. Really. For hosts, networks, ports, ...

Don't know if that is helpful but it definitely works for me to keep a complex and grown over years configuration readable.

HTH,
Patrick