No WAN connection unless activating Far Gateway option

[quantumjohnny]

Hi,

I set up a pretty standard OPNsense config behind a router from my ISP.
WAN interface (igb3) points to the router and gets IP via DHCP. IP is optained as 192.168.1.3, Router is set as Gateway at 192.168.1.1
LAN interface (igb2) has my laptop and assigns IPs via Kea DHCP (10.0.1.0/24) subnet.

From my laptop I can access the OPNsense host nicely, everything on the LAN side seems to work.

However, from the OPNsense host I have no access towards the router (I still believe that it was working nicely until recently, so maybe the upgrade to 24.1.10 changed something).

So I could not ping my ISP router (192.168.1.1), getting

Code: [Select]

PING 192.168.1.1 (192.168.1.1): 56 data bytes                                                                                                                                                                                             
ping: sendto: Invalid argument

Anything outside my local net (LAN) cannot be pinged. Unbound also didnt work (unsurprisingly). The only somewhat unhelpful message I found in the logs was:

Code: [Select]

arpresolve: can't allocate llinfo for 192.168.1.1 on igb3

With the above error I came to set the gateway to "Far Gateway" which is supposed to be for This will allow the gateway to exist outside of the interface subnet.
But my Gateway at 192.168.1.1 is in the WAN subnet of 192.168.1.3/24 ?

From the setup of routes and gateway everything looks fine as far as I can tell.
ifconfig

Code: [Select]

igb2: flags=8863<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
        description: LAN (lan)
        options=4802028<VLAN_MTU,JUMBO_MTU,WOL_MAGIC,NOMAP>
        ether 00:0d:b9:61:ad:f6
        inet 10.0.1.1 netmask 0xffffff00 broadcast 10.0.1.255
        media: Ethernet autoselect (1000baseT <full-duplex>)
        status: active
        nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
igb3: flags=8863<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
        description: WAN (wan)
        options=4802028<VLAN_MTU,JUMBO_MTU,WOL_MAGIC,NOMAP>
        ether 00:0d:b9:61:ad:f7
        inet 192.168.1.3 netmask 0xffffff00 broadcast 192.168.1.255
        media: Ethernet autoselect (1000baseT <full-duplex>)
        status: active
        nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>

netstat -rn

Code: [Select]

Destination        Gateway            Flags     Netif Expire
default            192.168.1.1      UGS        igb3
10.0.1.0/24        link#3             U          igb2
10.0.1.1           link#3             UHS         lo0
127.0.0.1          link#6             UH          lo0
192.168.1.3      link#4             UHS         lo0

Setting the "Far Gateway" option for the configured gateway adds one entry to the above routes:

Code: [Select]

192.168.1.1      link#4             UHS        igb3

I am at a loss why this is happening and why I have to set the "Far Gateway" option when my Gateway appears to be in the same subnet as the WAN interface.

Related:
https://forum.opnsense.org/index.php?topic=11991.0
But for me manually saving the WAN interface configuration did not change anything.

Possibly related:
https://forum.opnsense.org/index.php?topic=34340.0

[franco]

Something is off here... you say its 192.168.1.1 but its 192.168.178.1 which makes sense then the way you configured it correctly with far gateway checkbox.


Cheers,
Franco

[quantumjohnny]

Sorry I just wanted to obscure the subnet. That is 192.168.178.0/24. But shouldn't matter. So everything here was from that subnet. Fixed in the original post.

[franco]

Still not sure:

> arpresolve: can't allocate llinfo for 192.168.1.1 on igb3

This only ever happens when the attached subnet / netmask disagrees about it for a valid (but sometimes obscure) reason.


Cheers,
Franco

[quantumjohnny]

This only ever happens when the attached subnet / netmask disagrees about it for a valid (but sometimes obscure) reason.

Fully agree that it appears that way. But I don't see anywhere that this is the case. And it is rather strange considering I did not change much of the defaults here. Any idea where I can dig deeper?

[franco]

Any bridged ports on the OPNsense or other devices attached to WAN? Some sort of "intelligent" switch there?


Cheers,
Franco

[quantumjohnny]

Nothing there. No bridges or other interfaces. Anything fancy should also show up in the routing table, though.

After the n-th reboot it now works even without the "Far Gateway" option enabled. I did not change anything aside from that option. Will see how it develops.

Anyway, would you have a hint on how to debug a failing route when the routing table seems correct?

I'll update this if anything new pops up.

[franco]

Strange... the only thing I could think of is a bad DHCP offer that's no longer traceable. Maybe the router option was mismatching or missing or the gateway had a manual router IP at one point for testing?


Cheers,
Franco