WAN interface flapping with 22.1.2

[tracerrx]

If something is fixed in 13.1, that's not currently in 13, let @franco know which commit (That's what he is asking above). If you search through the recent commits to FreeBSD 13.1 https://github.com/freebsd/freebsd-src/commits/releng/13.1 there are a quite a few that mention ethernet drivers and/or MAC addresses.  But I don't see anything that looks like it effects the particular issues we've been experiencing in this thread.

[atxx]

I registered in order to note that I'm experiencing this issue on Broadcom BCM5720 (Microserver Gen8) therefore this is probably not Intel specific as speculated.

I updated to 22.1.6 three days ago and my WAN interface started flapping next day.
After a restart it was fine, until two days later (today) when the flapping started happening again.
Issue still present on 22.1.7_1.

IPS is on, mac spoofing is active, and gateway monitor is disabled. ifdown/up doesn't solve the issue, I need to restart the device for this to get resolved; this issue is quite disruptive.

I do feel obliged to say that Opnsense has pretty much been smooth sailing up until now. Thank you for your good work!

[Grossartig]

I updated to 22.1.6 three days ago and my WAN interface started flapping next day.
After a restart it was fine, until two days later (today) when the flapping started happening again.
Issue still present on 22.1.7_1.

IPS is on, mac spoofing is active, and gateway monitor is disabled. ifdown/up doesn't solve the issue, I need to restart the device for this to get resolved; this issue is quite disruptive.

Can you see if it's more stable when IPS is disabled? I had to disable it on my end for WAN to stabilize.

[tracerrx]

IPS is on, mac spoofing is active, and gateway monitor is disabled. ifdown/up doesn't solve the issue, I need to restart the device for this to get resolved; this issue is quite disruptive.

Are you running IPS on the LAN or WAN?

[atxx]

@Grossartig After I posted, I contacted my ISP and gave them my opnsense device's MAC address therefore I turned mac spoofing off. If the issue appears again I'll try turning IPS off and report back.

@tracerrx WAN

[Vesalius]

Is moving to 13.1 a possibility for 22.7? Assuming that this issue would be resolved?

using 13.1 for 22.7 is the plan regardless.

https://forum.opnsense.org/index.php?topic=28302.msg137463#msg137463

[bugvito]

Good find Vesalius,

I had time to test a bit last night, and the steps posted to compile the IGB drivers did solve my WAN problems. I have another issue so my testing only lasted about 30-60 minutes, but without any WAN issues.

I will remove the tunable setting specifying the compiled IGB driver, and potentially test 22.7 if all goes well.

None of this helps identifying/resolve the real cause, unfortunately.

Thanks!

[subivoodoo]

I did a quick test with 22.7.pre3 as described in this post... it doesn't seem to work for me  (MAC spoofing + IPS).

Console screenshots attached... before and after the same issue.

[tracerrx]

@bugvito you will need to re-compile the IGB driver again for 22.7 (and for every kernel update).  What brand hardware are you using?

[Dantichrist]

@tracerrx

I finally got some time to play with updating the NIC driver, and it works like a charm. I've had the MAC spoofed for about 12 hours now with no issues at all. Before the interface would drop within 15 to 30 min.

I just wanted to thank you for posting a fix!

[tracerrx]

@dantichrist you will need to re-compile the IGB driver again for 22.7 (and for every kernel update there after).  What brand hardware are you using?

[Dantichrist]

@tracerrx It's just an old i7-4790 box that I'm using. I'm using a dual intel 82576 (igb) card in it/bare metal. No vlans or anything fancy but I am running suricata/zenarmor. I'm planning on doing this after each update until the driver in BSD is sorted out. Now I just have to get os-ddclient working with godaddy and I'll be all set. 

[firewall]

after a pleasant 12 or so hours with the driver from intel i'm right back to square one--and its seemingly worse than before. also a pleasant few years with opnsense absolutely rekt by 2 months of unreliable connectivity.

[bugvito]

@bugvito you will need to re-compile the IGB driver again for 22.7 (and for every kernel update).  What brand hardware are you using?

I'm aware. I was wondering if I could find a way to execute a command on a new kernel install event to recompile the driver. With that said, the driver still loaded and worked fine under 22.7pre3 without recompiling, but is a lucky/risky thing to do/try. This would still be an unviable solution, as this may cause problems down the road.

I did disable my compiled igd driver (confirmed not loaded with kldstat) and updated to 22.7.pre3, and to my pleasant surprise, everything is working fine for me (after 1h).
The test box is a cheap HSUNG RS34g with 4xi225 (I believe that the board itself is popular: 1090np-12).

Unfortunately this does not help identifying the real issue at hand for this topic, and as some already reported, 22.7 did not resolve their issue, while the original issue may not be intel nic specific.

[tracerrx]

@bugvito you should only need to re-compile the driver when you get a kernel update. I would not disable the built in igb driver, if you want to stop using the newly compiled driver just change your tunable to "if_igb_updated_load=NO"