PF states created with route-to: 0.0.0.0 after boot, causing arpresolve errors

Started by geri441, Today at 04:59:30 PM

Previous topic - Next topic
Hi,

I am seeing what looks like a PF/gateway initialization issue after boot on OPNsense 26.7.2_2.

Environment
OPNsense 26.7.2_2

FreeBSD 15.1-RELEASE-p2
amd64

WAN uses an Intel ix interface with a static IPv4 gateway.

The firewall also has several internal interfaces/VLANs with rules that explicitly use the WAN gateway for Internet traffic.

Problem

Shortly after boot, the kernel repeatedly logs:

arpresolve: can't allocate llinfo for 0.0.0.0 on ix0

While this is happening, the routing table itself is correct.

For example:

default            <WAN_GATEWAY>       UGS      ix0

and:

route -n get 0.0.0.0

route to: 0.0.0.0
destination: 0.0.0.0
mask: 0.0.0.0
gateway: <WAN_GATEWAY>
interface: ix0
flags: <UP,GATEWAY,DONE,STATIC>

The currently loaded PF rules are also correct and contain:

route-to (ix0 <WAN_GATEWAY>)

There are no active rules containing:

route-to (ix0 0.0.0.0)

However, some PF states created shortly after boot contain:

route-to: 0.0.0.0@ix0

For example:

all udp <INTERNAL_HOST_A>:514 <- <INTERNAL_HOST_B>:514
    route-to: 0.0.0.0@ix0

I also found affected states for normal Internet traffic, so this is not limited to one internal destination.

Examples included HTTPS traffic to public IP addresses such as:

1.1.1.1:443
8.8.8.8:443

In the diagnostic capture there were several dozen states containing:

route-to: 0.0.0.0@ix0
Timing

The bad states appear very early after boot.

In one capture, system uptime was only a few minutes and the affected PF states were almost the same age, indicating they had been created roughly during the first minute after startup.

The arpresolve errors started afterward and continued while those states existed.

Test performed

Once the firewall was fully booted, I flushed the state table:

pfctl -F states

No firewall rules, aliases, gateways, NAT configuration, or interfaces were changed.

Immediately afterward:

pfctl -ss -vv | grep 'route-to: 0\.0\.0\.0@ix0'

returned no results.

New states were then recreated automatically by normal traffic and all of them used the correct gateway:

route-to: <WAN_GATEWAY>@ix0

For example:

route-to: <WAN_GATEWAY>@ix0
route-to: <WAN_GATEWAY>@ix0
route-to: <WAN_GATEWAY>@ix0

The kernel arpresolve messages also stopped. The last recorded message remained at the same kernel uptime timestamp even several minutes after the state flush.

I suspect there may be a boot-time ordering/race condition where some policy-routed states are created before the WAN gateway information is fully available to PF.

Has anyone else seen PF states being created with route-to: 0.0.0.0 during early boot on 26.7.x?