kernel: igb1: link state changes to DOWN / UP / DOWN

[hirschferkel]

Hi there,

I updated my 18.7.6 system today and since then my router goes wired.
I reinstalled the system, settings, changed the Network inputs (assigments of igb1 and igb2) cables, and so on. But it doesn't stop to break Internet connection. I thought maybe igb1 device is broken, but when I switched to igb2 it was the same result.

The whole system ran perfect for 3 month, and I do not know what I can do to get it work again. Any idea? I never saw such messages before.

All the best, hischrschferkel

Code Select Expand

Nov 7 18:51:35 opnsense: /usr/local/etc/rc.linkup: HOTPLUG: Configuring interface opt1
Nov 7 18:51:35 opnsense: /usr/local/etc/rc.linkup: DEVD Ethernet attached event for opt1
Nov 7 18:51:35 kernel: igb1: link state changed to DOWN
Nov 7 18:51:35 kernel: igb1: link state changed to UP
Nov 7 18:51:32 opnsense: /usr/local/etc/rc.linkup: Clearing states to old gateway 192.168.178.1.
Nov 7 18:51:32 opnsense: /usr/local/etc/rc.linkup: DEVD Ethernet detached event for opt1
Nov 7 18:51:31 kernel: igb1: link state changed to DOWN
Nov 7 18:51:31 kernel:

Code Select Expand

Nov 7 18:42:45 opnsense: /usr/local/etc/rc.newwanip: Interface '' is disabled or empty, nothing to do.
Nov 7 18:42:45 opnsense: /usr/local/etc/rc.newwanip: IP renewal is starting on 'ovpns1'
Nov 7 18:42:44 kernel: ovpns1: link state changed to UP
Nov 7 18:42:41 kernel: ovpns1: link state changed to DOWN
Nov 7 18:42:40 opnsense: /usr/local/etc/rc.openvpn: OpenVPN: Resync server1 VPN-ADMIN
Nov 7 18:42:40 opnsense: /usr/local/etc/rc.openvpn: OpenVPN: One or more OpenVPN tunnel endpoints may have changed its IP. Reloading endpoints that may use KABEL_DHCP.
Nov 7 18:42:39 opnsense: /usr/local/etc/rc.dyndns: Dynamic DNS (XXX.YYY.ZZZ): (Success) DNS hostname update successful.
Nov 7 18:42:39 opnsense: /usr/local/etc/rc.dyndns: Dynamic DNS: updating cache file /var/cache/dyndns_opt1_ixtract.ddns.net_0.cache: 91.XXX.128.YYY
Nov 7 18:42:38 opnsense: /usr/local/etc/rc.newwanip: ROUTING: skipping IPv6 default route
Nov 7 18:42:38 opnsense: /usr/local/etc/rc.newwanip: ROUTING: keeping current default gateway '192.168.178.1'
Nov 7 18:42:38 opnsense: /usr/local/etc/rc.newwanip: ROUTING: setting IPv4 default route to 192.168.178.1
Nov 7 18:42:38 opnsense: /usr/local/etc/rc.newwanip: ROUTING: no IPv6 default gateway set, assuming wan
Nov 7 18:42:38 opnsense: /usr/local/etc/rc.newwanip: ROUTING: IPv4 default gateway set to opt1
Nov 7 18:42:38 opnsense: /usr/local/etc/rc.newwanip: ROUTING: entering configure using 'opt1'
Nov 7 18:42:38 opnsense: /usr/local/etc/rc.newwanip: On (IP address: 192.XXX.178.YYY) (interface: KABEL[opt1]) (real interface: igb1).
Nov 7 18:42:38 opnsense: /usr/local/etc/rc.newwanip: IP renewal is starting on 'igb1'
Nov 7 18:42:38 opnsense: /usr/local/etc/rc.dyndns: Dynamic DNS (XXX.YYY.ZZZ): Processing dummy update on No-IP free account. IP temporarily set to 192.168.1.1
Nov 7 18:42:37 kernel: igb1: link state changed to UP
Nov 7 18:42:34 kernel: igb1: link state changed to DOWN
Nov 7 18:42:32 opnsense: /usr/local/etc/rc.linkup: ROUTING: skipping IPv6 default route

Code Select Expand

Nov 7 19:05:45 apinger: Starting Alarm Pinger, apinger(25841)
Nov 7 19:05:45 apinger: Exiting on signal 15.
Nov 7 19:04:59 apinger: Starting Alarm Pinger, apinger(32367)
Nov 7 19:04:59 apinger: Exiting on signal 15.
Nov 7 19:04:17 apinger: Starting Alarm Pinger, apinger(2188)
Nov 7 19:04:17 apinger: Exiting on signal 15.
Nov 7 19:03:27 apinger: alarm canceled: KABEL_DHCP(192.168.178.1) *** down ***
Nov 7 19:03:27 apinger: ALARM: KABEL_DHCP(192.168.178.1) *** down ***
Nov 7 19:02:30 apinger: Starting Alarm Pinger, apinger(22673)
Nov 7 19:02:29 apinger: Exiting on signal 15.
Nov 7 19:02:27 apinger: Starting Alarm Pinger, apinger(9813)
Nov 7 19:02:26 apinger: Exiting on signal 15.
Nov 7 19:02:14 apinger: Error while feeding rrdtool: Broken pipe
Nov 7 19:01:14 apinger: Starting Alarm Pinger, apinger(23449)
Nov 7 19:01:14 apinger: Exiting on signal 15.
Nov 7 19:00:20 apinger: Error while feeding rrdtool: Broken pipe
Nov 7 18:56:20 apinger: rrdtool respawning too fast, waiting 300s.
Nov 7 18:55:20 apinger: Error while feeding rrdtool: Broken pipe
Nov 7 18:54:20 apinger: Starting Alarm Pinger, apinger(15224)
Nov 7 18:54:20 apinger: Exiting on signal 15.

[franco]

Hi hirschferkel,

Try to set "prefer dpinger" under Firewall: Settings: Advanced. It'll be the sole monitoring services as apinger will be removed.

Doesn't look like the update did this per se... Your WAN was jumpy.

> Error while feeding rrdtool: Broken pipe

This doesn't happen with dpinger.


Cheers,
Franco

[hirschferkel]

Dear Franco,

thanks a lot for your help, but actually this did not help anyway... I'm really clueless what to do, now. The whole system was absolutly stable and now I will always get this routing breaks. Any idea for that?

Code Select Expand

Nov 8 08:07:19 opnsense: /usr/local/etc/rc.newwanip: IP renewal is starting on 'igb2'
Nov 8 08:07:18 opnsense: /usr/local/etc/rc.linkup: HOTPLUG: Configuring interface opt1
Nov 8 08:07:18 opnsense: /usr/local/etc/rc.linkup: DEVD Ethernet attached event for opt1
Nov 8 08:07:17 opnsense: /usr/local/etc/rc.linkup: Clearing states to old gateway 192.168.178.1.
Nov 8 08:07:17 opnsense: /usr/local/etc/rc.linkup: DEVD Ethernet detached event for opt1
Nov 8 08:07:14 opnsense: /usr/local/etc/rc.linkup: ROUTING: skipping IPv6 default route
Nov 8 08:07:14 opnsense: /usr/local/etc/rc.linkup: ROUTING: keeping current default gateway '192.168.178.1'
Nov 8 08:07:14 opnsense: /usr/local/etc/rc.linkup: ROUTING: setting IPv4 default route to 192.168.178.1
Nov 8 08:07:14 opnsense: /usr/local/etc/rc.linkup: ROUTING: no IPv6 default gateway set, assuming wan
Nov 8 08:07:14 opnsense: /usr/local/etc/rc.linkup: ROUTING: IPv4 default gateway set to opt1
Nov 8 08:07:14 opnsense: /usr/local/etc/rc.linkup: ROUTING: entering configure using 'opt1'
Nov 8 08:07:13 opnsense: /usr/local/etc/rc.newwanip: Resyncing OpenVPN instances for interface KABEL.
Nov 8 08:07:10 opnsense: /usr/local/etc/rc.newwanip: Gateways status could not be determined, considering all as up/active. (Group: GatewayKundV)
Nov 8 08:07:10 opnsense: /usr/local/etc/rc.newwanip: ROUTING: skipping IPv6 default route
Nov 8 08:07:10 opnsense: /usr/local/etc/rc.newwanip: ROUTING: keeping current default gateway '192.168.178.1'
Nov 8 08:07:10 opnsense: /usr/local/etc/rc.newwanip: ROUTING: setting IPv4 default route to 192.168.178.1
Nov 8 08:07:10 opnsense: /usr/local/etc/rc.newwanip: ROUTING: no IPv6 default gateway set, assuming wan
Nov 8 08:07:10 opnsense: /usr/local/etc/rc.newwanip: ROUTING: IPv4 default gateway set to opt1
Nov 8 08:07:10 opnsense: /usr/local/etc/rc.newwanip: ROUTING: entering configure using 'opt1'
Nov 8 08:07:09 opnsense: /usr/local/etc/rc.newwanip: On (IP address: 192.168.178.60) (interface: KABEL[opt1]) (real interface: igb2).
Nov 8 08:07:09 opnsense: /usr/local/etc/rc.newwanip: IP renewal is starting on 'igb2'
Nov 8 08:07:09 kernel: igb2: link state changed to UP
Nov 8 08:07:04 opnsense: /usr/local/etc/rc.linkup: HOTPLUG: Configuring interface opt1
Nov 8 08:07:04 opnsense: /usr/local/etc/rc.linkup: DEVD Ethernet attached event for opt1
Nov 8 08:07:04 kernel: igb2: link state changed to DOWN
Nov 8 08:07:04 kernel: igb2: link state changed to UP
Nov 8 08:07:01 opnsense: /usr/local/etc/rc.linkup: Clearing states to old gateway 192.168.178.1.
Nov 8 08:07:01 opnsense: /usr/local/etc/rc.linkup: DEVD Ethernet detached event for opt1
Nov 8 08:07:00 kernel: igb2: link state changed to DOWN

[hirschferkel]

Is it possible that it's a hardware defect within the router? Is there a way to somehow verify this?

I tried a backuped 18.1 version and had the same issues. So it is no problem of the 18.7 update...
The router gets internet access via a Fritzbox. As I mentioned the whole setup worked for 3 month and with this Firtzbox for 1 week, now... and suddenly the WAN begun flapping...

Best, hirschferkel

[franco]

I don't think this is related to software if the issue persists.

It may be faulty hardware, cables, switches, a jumpy ISP, even an ISP uplink router firmware update. Link up down come from the kernel (no changes here since 2018 from our side) and the kernel takes the UP / DOWN from the hardware which is attached via cable which speaks to a network port on the other side.


Cheers,
Franco

[hirschferkel]

Hi Franco,

I changed all ports / cables / devices...
It's abolutely not possible that all these elements are broken.

There is an issue documented on Github stating a similar (as I think) issue since May! So maybe it is a software problem?
https://github.com/opnsense/core/issues/2244
Best, hirschferkel

[hirschferkel]

And it seems I'm still not the only one who is suffering this issue...
https://forum.opnsense.org/index.php?topic=8975.0

[hirschferkel]

I don't think this is related to software if the issue persists.

So it is definitely a software problem. I took a different router in front of my OPNsense router (In this case it replaced a Fritz!box with a O2 Homebox) and the router works fine again! No errors at all.

So there is a problem with OPNsense when it is connected to another router where it gains internet access, somehow the WAN breaks down. But what can I do to make it work again, do you have any idea?


Best, hirschferkel

[hirschferkel]

I changed the port from the Fritz!box to "exposed host" and now the OPNsense router is running again. Don't understand what OPNsense negotiates with the Fritz!box in DHCP mode, but it definitely gets something wrong.

[jcrawford]

Did this get resolved, I believe I may be experiencing a similar issue:

https://forum.opnsense.org/index.php?topic=11879.0