OPNsense 2.5G WAN Troubleshooting - WAN Identifier and DHCP

Started by Drid, May 09, 2025, 04:23:35 AM

Previous topic - Next topic
TL;DR: OPNsense freezes on a 2.5G Windstream ONT only when the interface is named "WAN" possibly because DHCP lease. 
This is not a general hardware/driver issue, but seems to be an OPNsense software bug related to how it handles the "WAN" identifier and DHCP.  Help requested.

Problem:
Changed ISP from Spectrum to Windstream.
Unplugged Spectrum from WAN igc1 interface, plugged in Windstream.
OPNsense firewall/router experiences a WAN connectivity freeze after a few minutes when using an Intel i226-v NIC connected to a 2.5G Windstream ONT.
enabling/disabling igc1 restores service, for a few minutes, until it freezes again.
Plugging Spectrum back into igc1, service is restored and stable.
Created new interface named Windstream igc2, plugged windstream ONT into interface, service is rock solid.
re-assigned WAN to igc2, freezing begins again.

Tried igc3 as WAN same results as above.



System Details:

OPNsense version: 25.1.6_2-amd64
Hardware: CWWK
NIC: Intel i226-v (igc interfaces)
ONT: Windstream 2.5G ONT (single Ethernet port)
Cable Modem: Spectrum
Symptoms:

igc1 (or any i226-v interface) assigned as "WAN" and connected to Windstream ONT: Traffic passes for a few minutes, then the connection freezes, requiring igc1 interface restart.
igc1/igc2/igc3 (i226-v interfaces) connected to the same Windstream ONT, when NOT assigned as "WAN": No freezing occurs; the connection is stable.
igc1 (i226-v) assigned as "WAN" and connected to a Spectrum ISP: No freezing occurs; the connection is stable.
Firewall rules and interface configurations are identical for all igc interfaces in OPNsense.

Troubleshooting Steps Taken:


Cable Testing: Multiple known-good Cat5e/Cat6 cables have been tested, including the same cable used successfully with other interfaces and ISPs. Cables are not the issue.
Tried putting a dumb switch between my opnsense box and the ONT. Same results.
ONT Port: The Windstream ONT has only one Ethernet port, eliminating port-specific issues on the ONT.
OPNsense Port: All i226-v ports on OPNsense work perfectly with a different ISP (Spectrum) and when connected to the Windstream ONT, as long as they are not the designated "WAN" interface.
Speed/Duplex: Forced speed and duplex (2.5Gbps full-duplex) have been tested on both OPNsense and the Windstream ONT (where possible), with no change in behavior. Autonegotiation has also been tested.
Interface Reassignment: The issue follows the OPNsense "WAN" identifier. When igc2 was assigned as the WAN interface, the freezing issue occurred on igc2. When Windstream was moved to igc2, connectivity was restored.
Gateway settings are the same for both windstream and wan interfaces.

DHCP Lease/Renewal: The issue appears to be related to DHCP lease renewals?
The freezing appears to occur when /usr/local/etc/rc.newwanip is executed on an interface designated as "WAN." Interfaces with other names (e.g., "WINDSTREAM") do not exhibit this behavior.

A specific igc driver bug in OPNsense/FreeBSD that is only triggered when the interface is designated as "WAN" and a DHCP lease is renewed????
A configuration management issue within OPNsense, where settings are not correctly applied to i226-v interfaces when they are the "WAN" interface and a DHCP client is in use??
A problem with how OPNsense services (specifically DHCP-related services) interact with i226-v interfaces designated as "WAN."??
A rare OPNsense bug related to this specific hardware and ISP combination??

Any thoughts would be appreciated.

Bottom line, is there some special coding implemented for interfaces with internal identifier of WAN.

Does it matter from a performance or security perspective if I just never use the "wan" interface and continue to use the interface I created for Windstream?


Quote from: Drid on May 09, 2025, 04:23:35 AMDoes it matter from a performance or security perspective if I just never use the "wan" interface and continue to use the interface I created for Windstream?

No. (AFAIK)

Thanks newsense.

Heres what the general log shows when this occurs. Note, for this test, WAN was igc2.

2025-05-08T19:00:54-04:00   Notice   dhclient    dhclient-script: Creating resolv.conf
2025-05-08T19:00:54-04:00   Notice   dhclient    dhclient-script: New Hostname (igc2): OPNsense
2025-05-08T19:00:54-04:00   Notice   dhclient    dhclient-script: Reason RENEW on igc2 executing

Then traffic stops flowing.

If its interface Windstream, the same behaviour occurs, but traffic remains stable. It appears to occur every 30 minutes with zero issues on any interface not named WAN as its internal identifier.

@franco will certainly be interested in this issue. Your workaround should help narrow down the issue.

For further testing, have you tried renaming to wan rather than WAN ?

> Bottom line, is there some special coding implemented for interfaces with internal identifier of WAN.
No. It's just a label. All the workings AFAIK are exactly the same for any assigned label. What matters is the configuration of the "Device" in OPN language, and the drivers used. Same type of interface igc so that's not it. It's most likely therefore to be a difference in configuration settings that you haven't yet spotted.

Quote from: vk2him on May 09, 2025, 11:22:35 AMFor further testing, have you tried renaming to wan rather than WAN ?

It's labeled "wan", Im just used to typing WAN for 25 years.

I cannot spot a config difference. I've reviewed

Gateways
Interfaces
FW rules.

My config settings are dead simple and the fact that the wan interface is functional briefly, and also returns to functioning briefly after an interface reset makes me question its a slight config diff. I would prefer it was something on my end, would make the fix easier.

I've backed up my xml config and diff'd. I see no differences besides the naming.

I am going to perform a fresh install sometime next week, how that would make a difference I cannot say.

It's just very odd that if I plug back in the Spectrum ISP ethernet into wan, I have no issue. You would think that would point to the ISP, but the crux of this issue is if I plug the Windstream ISP into a fresh interface named something other than "wan" its stable.

It's why I was asking if there was some secret sauce for internal identifier's.

Appreciate the replies!