slow dhcp on wan -> broken NAT

[synfinatic]

This was happening on 26.1.x and continues on 26.7.1.   I'm on AT&T fiber and using an AZORES WAS-110 fiber module/ONT to bypass the AT&T provided gateway.  For this to work, my `ixl0` interface uses DHCP/DHCPv6 to get IP addresses and I create a virtual IP (192.168.11.10/24) on `ixl0` in order to manage the WAS-110 module.

The problem is that for whatever reason, getting an IP address from AT&T is _slow_.  After rebooting the firewall, I often have to login and manually run `dhclient ixl0` to force it to try again and that works.  Once an IPv4 address is obtained, DHCPv6 seems to happen automatically.

The problem is that the system seems to not really register that an IPv4 address has been obtained and so NAT is broken- running `tcpdump -ni ixl0` shows traffic egressing with a source IP of 192.168.11.10- not the public IP that AT&T has given via DHCP.  I've figured out that I can fix this by going into the interface config setting and re-applying the existing settings, but this sucks.

Seems like the root cause of the issue is AT&T being slow to give out an IP address over DHCP- any way to make things more robust?

[synfinatic]

Meant to mention that for outbound NAT I'm using "Manual outbound NAT rule generation" and selecting NAT Address: "interface address"

[drosophila]

You are perfectly right: NAT uses the first IP address it finds, which will usually be the first the interface gets assigned. So if you assign the 192.168.11.10 as "virtual" but the primary assignment fails on first try, then, even if DHCP eventually succeeds, 192.168.11.10 will be first in the list and NAT will use that (check ifconfig output for the order, the GUI overview may lie about it). It can't know whether you want to NAT inter-LAN or towards the internet, after all, and there is no way to tell it.
I've had a similar issue and could fix it only by running a cron job that reorders the addresses every minute (the smallest interval it allows to use). Sucks big time.
In your case, it should be possible to hook into the "newwanip" trigger, since it gets a new WAN IP, which this trigger is all about. You'd still need to have a script for it and manually configure the trigger from the command line. Either way, it'll break on a fresh install because none of this gets stored in the config. :(

[synfinatic]

So looking again at the NAT settings, seems like there are 3 options in the drop down for `ixl0`:

* interface address
* ONTWAN address -- ONTWAN is the name for ixl0
* 192.168.11.10 ONT Management -- the virtual IP

Seems like using the 2nd option should provide the correct results?   Or are the 1st and 2nd effectively the same option with different names?

[lmoore]

Another thought you could try.

- Assuming you have configured a static IP address on your ONT, add a static route on your ONT to reach your management network (LAN) via 192.168.11.10 (optional)

- Add a Bridge interface and include WAN as a member

- Assign the bridge interface and enter ONT in the description field

- Refresh the web page

- Select [ONT] interface and select Enable Interface

- Set IPv4 configuration to Static IPv4

- Set IPv4 address to 192.168.11.10/24

- Save the settings

- Apply the changes

- Select your [WAN] interface

- Check Block private networks

- Save the settings

- Apply the changes

- Review your firewall rules allowing connections to the 192.168.11.0/24 network

- On ONT, if no static route to your management network (LAN) - see above, create a Source NAT rule to the ONT network using the bridge address

[lmoore]

Firewall and Source NAT rules attached - note the use of tags.

[meyergru]

@lmoore: That will not cut it. The problem is not that the modem cannot be reached.

The problem is that the normal WAN NAT rule does not work, because the translated NAT IP is the virtual IP and not the real WAN IP.

This is, because when the DHCP client on WAN times out on the first try, the virtual IP gets to take the "primary" address slot on the WAN interface. When DHCP later succeeds, this will not change, such that the virtual IP will still be used for normal unbound NAT (not for the NAT rule that you need to reach the modem).

This problem does not manifest when the WAN is on another interface than the modem, like with PPPoE or with a VLAN or when DHCP is fast enough to succeed before the virtual IP gets initialized.

A fix for this is probably non-trivial, because you would have to delete and re-create the virtual IP when the DHCP finally succeeds and that could break other things.

The easy way out would be to skip modem access completely under these conditions. Another way would be if the modem could be configured such that the GUI can be configured on another VLAN than the internet connection. If the ISP wants a VLAN, you should tag it on OpnSense if you can and not leave that to the modem. There seems to be a community firmware for that XGS-PON:

https://pon.wiki/guides/masquerade-as-the-orange-sa-livebox-7-with-the-was-110/#from-the-web-ui

And you probably used the "fix-vlans" setting from here:

https://pon.wiki/guides/masquerade-as-the-att-inc-bgw320-500-505-with-the-was-110/#configure-ont-settings

I think (aka "IDK") that AT&T needs a VLAN. As I said, when you let OpnSense do the tagging, it could work, because you would use different interfaces for ONT and WAN, then.