After update, problems with LAN/WAN Interfaces

[Bytechanger]

Hi,

my OPNSense was running good.
But after update, problems with lan/ wan interfaces.

ZOTAC ZBOX-CI329NANO.CI

After minutes, hours suddenly dhcp from isp goes down.
Somtimes internetconnection goes down.
Tonight LAN connection goes down.

Driver issue?

Code: [Select]

pciconf -lv re1
re1@pci0:2:0:0: class=0x020000 rev=0x0c hdr=0x00 vendor=0x10ec device=0x8168 subvendor=0x10ec subdevice=0x0123
    vendor     = 'Realtek Semiconductor Co., Ltd.'
    device     = 'RTL8111/8168/8411 PCI Express Gigabit Ethernet Controller'
    class      = network
    subclass   = ethernet

 $ pciconf -lv re0
re0@pci0:1:0:0: class=0x020000 rev=0x0c hdr=0x00 vendor=0x10ec device=0x8168 subvendor=0x10ec subdevice=0x0123
    vendor     = 'Realtek Semiconductor Co., Ltd.'
    device     = 'RTL8111/8168/8411 PCI Express Gigabit Ethernet Controller'
    class      = network
    subclass   = ethernet


HArdware CRC, Hardware TSO, Hardware LRO, VLAN Hardware Filtering is off in config.

Greets

Byte

[chtse53]

You are using realtek NICs.  I suppose you have followed the instructions brefore upgrading: "The Realtek vendor driver is no longer bundled with the updated FreeBSD kernel.  If unsure whether FreeBSD 13 supports your Realtek NIC please install the os-realtek-re plugin prior to upgrading to retain operability of your NICs."

[Bytechanger]

OK, now I´m on 22.1, but there is no os-realtek-re plugin ?!
What to do?

Greets

Byte

[franco]

> OK, now I´m on 22.1, but there is no os-realtek-re plugin ?!

Errr what?!


Cheers,
Franco

[Bytechanger]

chtse53 said,

You are using realtek NICs.  I suppose you have followed the instructions brefore upgrading: "The Realtek vendor driver is no longer bundled with the updated FreeBSD kernel.  If unsure whether FreeBSD 13 supports your Realtek NIC please install the os-realtek-re plugin prior to upgrading to retain operability of your NICs."

So I´m on 22.1,  got trouble with lan/wan interfaces and look for os-realtek-re plugin.

Tonight at 01:00 I can´t reach ip of my OPNSense.
At 23:00 WAN get´s no DHCP fom ISP (DHCP down), and error so on after time working.
So I think, it could be driver problems with my realtek.
On 21.7 all good, after update to 22.1 errors.

Greets

Byte

[franco]

I'm sorry to say that "but there is no os-realtek-re plugin" is just not based in reality and the effort to double-check that is 10 seconds.


Cheers,
Franco

[Bytechanger]

OK, thanks.

But what could the solutions to my problem?
It seems realtek with 22.1 is not working probably?!

"The Realtek vendor driver is no longer bundled with the updated FreeBSD kernel"
Can I install the driver over ssh on my machine? How?"

Need help, please.

Greets

Byte

[Bytechanger]

So after 5 Hours same error, WAN and LAN down, OPNSense not reachable, reboot works...
my familiy is killing me.

So I think, this could be the syslog where error starts this time:

Code: [Select]

2022-01-29T10:14:38 Error opnsense /usr/local/etc/rc.newwanipv6: IPv6 renewal is starting on 're0'
2022-01-29T10:14:37 Error opnsense /usr/local/etc/rc.linkup: DEVD: Ethernet detached event for dynamic opt2(re1_vlan50)
2022-01-29T10:14:36 Error opnsense /usr/local/etc/rc.linkup: DEVD: Ethernet detached event for dynamic lan(re1)
Can anybody help please?

[fraenki]

Can anybody help please?

To to System -> Firmware -> Plugins, install "os-realtek-re", and reboot afterwards. It may improve the situation.
(Or use "pkg install os-realtek-re" when using a SSH session to install the driver.)

[Bytechanger]

OK, thanks, I think, I´ve overlookit at first time...

[DePingus]

OK, thanks, I think, I´ve overlookit at first time...

Hi. Did the realtek driver package solve your problems? If so, how has OPNsense been performing since? I have the same realtek chipset but haven't updated yet.

[Bytechanger]

Hi,

I ve had massive problems. After installing realtek plugin and setting tunables

Tunable: hw.re.msi_disable Value: 1
Tunable: hw.re.msix_disable Value: 1
Tunable: hw.re.eee_enable Value: 0
Tunable: hw.re.phy_power_saving Value: 0
Tunable: hw.re.phy_mdix_mode Value: 0

it works for me.
But now, I moved to proxmox on an host on intel nic.

Greets

Byte

[oh_lawd]

I have similar problems after 22.1 upgrade with Intel hardware:

Code: [Select]

igb0@pci0:1:0:0: class=0x020000 rev=0x03 hdr=0x00 vendor=0x8086 device=0x1533 subvendor=0xffff subdevice=0x0000
    vendor     = 'Intel Corporation'
    device     = 'I210 Gigabit Network Connection'
    class      = network
    subclass   = ethernet
Usually happens when I'm not home, and my family is (as they should) angry for internet not working. Router reboot works, but I haven't got my fingers into the logs yet. Last night it seemed to happen again, killing LAN completely for the house and as a side-effect the automated heating did not turn on. 21.x has been rock-solid and never had any issues with it.

[oh_lawd]

Trying to parse the logs and my Grafana instance, I see the network went down 06:33 AM, and the logs show normal WAN operations from the router, but LAN being down from that point on. Funny enough what I also noticed is when triggering a reboot, the LAN started working immediately when the shutdown sequence started until the power was off.

Also, interestingly, I see this hourly spike in the state table usage. The LAN went down during one of these peaks (pf.entries):

[oh_lawd]

This morning again the whole LAN went down. It seems to happen quite often now. I tried a few settings from the earlier posts:

Code: [Select]

sysctl dev.igb.0.eee_control=0
sysctl dev.igb.1.eee_control=0

Disabling energy saving just turned off the network interfaces and I needed to reboot.

Code: [Select]

dev.igb.1.iflib.disable_msix=1
dev.igb.0.iflib.disable_msix=1

If disabling msi-x helped with the realtek drivers, maybe it'll help here too. These settings didn't break the connection, so let's see if they have any effect to the problem.

Here's the full output of pciconf:

Code: [Select]

igb0@pci0:1:0:0: class=0x020000 rev=0x03 hdr=0x00 vendor=0x8086 device=0x1533 subvendor=0xffff subdevice=0x0000
    vendor     = 'Intel Corporation'
    device     = 'I210 Gigabit Network Connection'
    class      = network
    subclass   = ethernet
    cap 01[40] = powerspec 3  supports D0 D3  current D0
    cap 05[50] = MSI supports 1 message, 64 bit, vector masks
    cap 11[70] = MSI-X supports 5 messages, enabled
                 Table in map 0x1c[0x0], PBA in map 0x1c[0x2000]
    cap 10[a0] = PCI-Express 2 endpoint max data 256(512) FLR RO NS
                 max read 512
                 link x1(x1) speed 2.5(2.5) ASPM disabled(L0s/L1)
    ecap 0001[100] = AER 2 0 fatal 0 non-fatal 0 corrected
    ecap 0003[140] = Serial 1 00f421ffff6833bb
    ecap 0017[1a0] = TPH Requester 1