OPNsense Forum

Archive => 22.7 Legacy Series => Topic started by: firewall on August 02, 2022, 11:29:34 pm

Title: WAN Flapping Addressed with 22.7?
Post by: firewall on August 02, 2022, 11:29:34 pm
The (https://forum.opnsense.org/index.php?topic=27299.0) upgrade (https://forum.opnsense.org/index.php?topic=29571.0) to (https://forum.opnsense.org/index.php?topic=29601.0) 22.x (https://forum.opnsense.org/index.php?topic=29590.0) earlier (https://forum.opnsense.org/index.php?topic=28791.0) this (https://forum.opnsense.org/index.php?topic=28610.0) year (https://forum.opnsense.org/index.php?topic=28127.0) was (https://forum.opnsense.org/index.php?topic=28460.0) accompanied (https://forum.opnsense.org/index.php?topic=28515.0) by (https://forum.opnsense.org/index.php?topic=26801.0) rather (https://forum.opnsense.org/index.php?topic=28158.0) severe (https://forum.opnsense.org/index.php?topic=26589.0) WAN (https://forum.opnsense.org/index.php?topic=27419.0) connectivity (https://forum.opnsense.org/index.php?topic=27705.0) issues (https://forum.opnsense.org/index.php?topic=27623.0) experienced (https://forum.opnsense.org/index.php?topic=27051.0) by (https://forum.opnsense.org/index.php?topic=27480.0) many (https://forum.opnsense.org/index.php?topic=27337.0) forum (https://forum.opnsense.org/index.php?topic=27400.0) users (https://forum.opnsense.org/index.php?topic=27162.0) and (https://forum.opnsense.org/index.php?topic=27161.0) often (https://forum.opnsense.org/index.php?topic=27277.0) echoed (https://forum.opnsense.org/index.php?topic=27270.0) by (https://forum.opnsense.org/index.php?topic=26929.0) others. (https://forum.opnsense.org/index.php?topic=29594.0) These (https://forum.opnsense.org/index.php?topic=26587.0) are (https://forum.opnsense.org/index.php?topic=26573.0) unique (https://forum.opnsense.org/index.php?topic=26680.0) posts (https://forum.opnsense.org/index.php?topic=26652.0) from (https://forum.opnsense.org/index.php?topic=29556.0) 22.x. (https://forum.opnsense.org/index.php?topic=26554.0)

Though admittedly a few of the links above are only possibly related to the underlying issue I'm sure I didn't track down 100% of the threads that were.

Given the extent of my digging I trust it's evident that this has been a major thorn. Regardless, I've stuck with OPNsense with the hopes that a fix would arrive eventually.

Question: noting the handful of interface and dhcpd items in the changelog for 22.7, were any of them intended to address this issue? If not, did the 22.7 release unwind any related changes that may have been introduced with 22.1?
Title: Re: WAN Flapping Addressed with 22.7?
Post by: franco on August 03, 2022, 09:32:02 am
To be frank, clustering loosely related reports, answered and unanswered threads for 22.1 and 22.7 and solved issues with 22.7 I'm not sure what I should be looking at.

Wait for 22.7.1 and upgrade. It's going to be ok.


Cheers,
Franco
Title: Re: WAN Flapping Addressed with 22.7?
Post by: Dantichrist on August 04, 2022, 11:32:21 pm
The issue with MAC spoofing that's in this thread https://forum.opnsense.org/index.php?topic=27299.0 (https://forum.opnsense.org/index.php?topic=27299.0) still persists with an IGB/Intel 82576 NIC.

I haven't looked into it much yet. The strange part is that the Intel NIC driver that's included in this ver is the current driver, and compiling/using the same driver (2.5.24) as described on page 6 in that thread will fix it.
Title: Re: WAN Flapping Addressed with 22.7?
Post by: firewall on September 20, 2022, 07:52:08 am
To be frank, clustering loosely related reports, answered and unanswered threads for 22.1 and 22.7 and solved issues with 22.7 I'm not sure what I should be looking at.

"loosely related" reports that all point to a common issue with wan connectivity; likely pertaining to intel nics. never acknowledged as a common issue and apparently one which users continued to experience for the duration of 22.1 series despite numerous purported workarounds.

i'm on 22.7.4 now and i still have the issue.

i don't know what to tell you that i haven't already (https://forum.opnsense.org/index.php?topic=27299.msg140020#msg140020) besides "it's not working the way it's supposed to".
Title: Re: WAN Flapping Addressed with 22.7?
Post by: franco on September 26, 2022, 10:36:38 am
> i'm on 22.7.4 now and i still have the issue.

That's interesting, because the main issue from 22.1 was fixed in this release so it may be some issue, but not most of what you quote.


Cheers,
Franco
Title: Re: WAN Flapping Addressed with 22.7?
Post by: meelokun on September 26, 2022, 10:47:26 pm
I have this issue as of last week, and its causing us (all the tenants) alot of pain - its flapping very regularly, and interupting alot of our sessions.

https://pastebin.com/rTyajzcD

Why is this happening?

I disabled Maltrail, incase it has something to do with putting the Lan port in promiscuous mode (saw it on the display output).
Title: Re: WAN Flapping Addressed with 22.7?
Post by: franco on September 27, 2022, 11:29:36 am
All I can see is the NIC is ordered to shut down:

/usr/local/etc/rc.linkup: DEVD: Ethernet detached event for dynamic wan(igc2)

So it goes into a cycle. Whether this is a permanent driver issue (igc is pretty new and unsupported by Intel on FreeBSD) or an issue in conjunction with netmap(4) use I do not know.

netmap(4) behaviour will improve in the mid-term, but I'm not authorised to say more ;)


Cheers,
Franco
Title: Re: WAN Flapping Addressed with 22.7?
Post by: meelokun on September 27, 2022, 03:10:52 pm
Disabling Maltrail deffinitely helped....

I dont understand - i had sensei/zenconsole (LAN IDS/IPS) working before (pre 22.7), and now i cant use any IDS/IPS now, as it causes this issue to resurface - whether it be Maltrail or Sensei/Zenconsole.

Perhaps this is related to how it puts the port in promiscuous mode?
Title: Re: WAN Flapping Addressed with 22.7?
Post by: meelokun on September 27, 2022, 10:20:34 pm
welp it happened a couple of times today again, even after disabling Maltrail - looks like i was wrong in my theory of it being the IDS stuff...

https://pastebin.com/jm0MuA8H
Title: Re: WAN Flapping Addressed with 22.7?
Post by: franco on September 28, 2022, 08:45:39 am
Well, this is the NIC decision to shut down:

2022-09-26T16:37:20-04:00   Error   opnsense    /usr/local/etc/rc.linkup: The command '/sbin/dhclient -c '/var/etc/dhclient_wan.conf' -p '/var/run/dhclient.igc2.pid' 'igc2'' returned exit code '1', the output was 'igc2: no link .............. giving up'

Can you post dmesg output please?

FWIW, as soon as you use Zenarmor or IPS mode Suricata this can happen, but external switch issues, overload, etc. can cause this too. WAN flapping is especially serious as it recycles all connectivity on the box although it is what it is when it occurs. Finding the reason for it is not as easy as posting logs here. Sometimes ISP routers also overload and cause this. So the reaction to an issue is recorded by logs, but it's not the cause.


Cheers,
Frnaco