OPNsense
  • Home
  • Help
  • Search
  • Login
  • Register

  • OPNsense Forum »
  • English Forums »
  • 22.1 Legacy Series »
  • WAN interface flapping with 22.1.2
« previous next »
  • Print
Pages: [1] 2 3 ... 12

Author Topic: WAN interface flapping with 22.1.2  (Read 17726 times)

foxmanb

  • Newbie
  • *
  • Posts: 8
  • Karma: 0
    • View Profile
WAN interface flapping with 22.1.2
« on: March 03, 2022, 01:45:18 pm »
Hi All,
New here, and not a expert in linux/bsd etc, so please, go easy on me.. lol.

I upgraded to 22.1.2 on release day, and when my router came back, up, the WAN interface would pick up a DHCP address, hold it for about 10 seconds, then drop, renew, hold it for 10 seconds, then drop... rinse, wash, repeat. In those 10 seconds I was able to use the opnsense-revert command to roll back to 22.1.1. The interface is stable under the old version. I am running a custom built box, intel 9500T, Intel x550-t2 NIC. "Gateway" is a Motorola MB8611 cable modem which connects at 2.5G.

This is what I see in the logs over and over again with the new 22.1.2 version.

2022-03-03T07:03:46-05:00   Critical   dhclient   exiting.   
2022-03-03T07:03:46-05:00   Error   dhclient   connection closed   
2022-03-03T07:03:46-05:00   Error   opnsense   /usr/local/etc/rc.linkup: DEVD: Ethernet detached event for dynamic wan(ix1)   
2022-03-03T07:03:41-05:00   Error   opnsense   /usr/local/etc/rc.linkup: Warning! dhcpd_radvd_configure(auto) found no suitable IPv6 address on ix0   
2022-03-03T07:03:41-05:00   Error   opnsense   /usr/local/etc/rc.linkup: ROUTING: keeping current default gateway '73.134.218.1'   
2022-03-03T07:03:41-05:00   Error   opnsense   /usr/local/etc/rc.linkup: ROUTING: setting IPv4 default route to 73.134.218.1   
2022-03-03T07:03:41-05:00   Error   opnsense   /usr/local/etc/rc.linkup: ROUTING: IPv4 default gateway set to wan   
2022-03-03T07:03:40-05:00   Error   opnsense   /usr/local/etc/rc.linkup: ROUTING: entering configure using 'wan'   
2022-03-03T07:03:37-05:00   Error   opnsense   /usr/local/etc/rc.newwanip: Resyncing OpenVPN instances for interface WAN.   
2022-03-03T07:03:36-05:00   Error   opnsense   /usr/local/etc/rc.newwanip: The WAN_DHCP monitor address is empty, skipping.   
2022-03-03T07:03:36-05:00   Error   opnsense   /usr/local/etc/rc.newwanip: ROUTING: keeping current default gateway '73.134.218.1'   
2022-03-03T07:03:36-05:00   Error   opnsense   /usr/local/etc/rc.newwanip: ROUTING: setting IPv4 default route to 73.134.218.1   
2022-03-03T07:03:36-05:00   Error   opnsense   /usr/local/etc/rc.newwanip: ROUTING: IPv4 default gateway set to wan   
2022-03-03T07:03:36-05:00   Error   opnsense   /usr/local/etc/rc.newwanip: ROUTING: entering configure using 'wan'   
2022-03-03T07:03:35-05:00   Error   opnsense   /usr/local/etc/rc.newwanip: On (IP address: 73.134.x.x) (interface: WAN[wan]) (real interface: ix1).   
2022-03-03T07:03:35-05:00   Error   opnsense   /usr/local/etc/rc.newwanip: IPv4 renewal is starting on 'ix1'   
2022-03-03T07:03:34-05:00   Error   dhclient   send_packet: Network is down   
2022-03-03T07:03:29-05:00   Error   opnsense   /usr/local/etc/rc.linkup: DEVD: Ethernet attached event for dynamic wan(ix1)   
2022-03-03T07:03:28-05:00   Error   opnsense   /usr/local/etc/rc.linkup: Clearing states for stale wan route on ix1   
2022-03-03T07:03:28-05:00   Critical   dhclient   exiting.   
2022-03-03T07:03:28-05:00   Error   dhclient   connection closed   
2022-03-03T07:03:28-05:00   Error   opnsense   /usr/local/etc/rc.linkup: DEVD: Ethernet detached event for dynamic wan(ix1)   
2022-03-03T07:03:23-05:00   Error   opnsense   /usr/local/etc/rc.linkup: Warning! dhcpd_radvd_configure(auto) found no suitable IPv6 address on ix0   
2022-03-03T07:03:23-05:00   Error   opnsense   /usr/local/etc/rc.linkup: ROUTING: keeping current default gateway '73.134.218.1'   
2022-03-03T07:03:23-05:00   Error   opnsense   /usr/local/etc/rc.linkup: ROUTING: setting IPv4 default route to 73.134.218.1   
2022-03-03T07:03:23-05:00   Error   opnsense   /usr/local/etc/rc.linkup: ROUTING: IPv4 default gateway set to wan   
2022-03-03T07:03:22-05:00   Error   opnsense   /usr/local/etc/rc.linkup: ROUTING: entering configure using 'wan'   
2022-03-03T07:03:19-05:00   Error   opnsense   /usr/local/etc/rc.newwanip: Resyncing OpenVPN instances for interface WAN.   
2022-03-03T07:03:18-05:00   Error   opnsense   /usr/local/etc/rc.newwanip: The WAN_DHCP monitor address is empty, skipping.   
2022-03-03T07:03:18-05:00   Error   opnsense   /usr/local/etc/rc.newwanip: ROUTING: keeping current default gateway '73.134.218.1'   
2022-03-03T07:03:18-05:00   Error   opnsense   /usr/local/etc/rc.newwanip: ROUTING: setting IPv4 default route to 73.134.218.1   
2022-03-03T07:03:18-05:00   Error   opnsense   /usr/local/etc/rc.newwanip: ROUTING: IPv4 default gateway set to wan   
2022-03-03T07:03:18-05:00   Error   opnsense   /usr/local/etc/rc.newwanip: ROUTING: entering configure using 'wan'   
2022-03-03T07:03:17-05:00   Error   opnsense   /usr/local/etc/rc.newwanip: On (IP address: 73.134.x.x) (interface: WAN[wan]) (real interface: ix1).   
2022-03-03T07:03:17-05:00   Error   opnsense   /usr/local/etc/rc.newwanip: IPv4 renewal is starting on 'ix1'   
2022-03-03T07:03:12-05:00   Error   opnsense   /usr/local/etc/rc.linkup: DEVD: Ethernet attached event for dynamic wan(ix1)   
2022-03-03T07:03:11-05:00   Error   opnsense   /usr/local/etc/rc.linkup: Clearing states for stale wan route on ix1   
2022-03-03T07:03:11-05:00   Critical   dhclient   exiting.   
2022-03-03T07:03:11-05:00   Error   dhclient   connection closed   
2022-03-03T07:03:11-05:00   Error   opnsense   /usr/local/etc/rc.linkup: DEVD: Ethernet detached event for dynamic wan(ix1)   
2022-03-03T07:03:06-05:00   Error   opnsense   /usr/local/etc/rc.linkup: Warning! dhcpd_radvd_configure(auto) found no suitable IPv6 address on ix0   
2022-03-03T07:03:05-05:00   Error   opnsense   /usr/local/etc/rc.linkup: ROUTING: keeping current default gateway '73.134.218.1'   
2022-03-03T07:03:05-05:00   Error   opnsense   /usr/local/etc/rc.linkup: ROUTING: setting IPv4 default route to 73.134.218.1   
2022-03-03T07:03:05-05:00   Error   opnsense   /usr/local/etc/rc.linkup: ROUTING: IPv4 default gateway set to wan   
2022-03-03T07:03:05-05:00   Error   opnsense   /usr/local/etc/rc.linkup: ROUTING: entering configure using 'wan'   
2022-03-03T07:03:02-05:00   Error   opnsense   /usr/local/etc/rc.newwanip: Resyncing OpenVPN instances for interface WAN.   
2022-03-03T07:03:01-05:00   Error   opnsense   /usr/local/etc/rc.newwanip: The WAN_DHCP monitor address is empty, skipping.   
2022-03-03T07:03:01-05:00   Error   opnsense   /usr/local/etc/rc.newwanip: ROUTING: keeping current default gateway '73.134.218.1'   
2022-03-03T07:03:01-05:00   Error   opnsense   /usr/local/etc/rc.newwanip: ROUTING: setting IPv4 default route to 73.134.218.1   
2022-03-03T07:03:01-05:00   Error   opnsense   /usr/local/etc/rc.newwanip: ROUTING: IPv4 default gateway set to wan   
2022-03-03T07:03:01-05:00   Error   opnsense   /usr/local/etc/rc.newwanip: ROUTING: entering configure using 'wan'   
2022-03-03T07:03:00-05:00   Error   opnsense   /usr/local/etc/rc.newwanip: On (IP address: 73.134.x.x) (interface: WAN[wan]) (real interface: ix1).   
2022-03-03T07:03:00-05:00   Error   opnsense   /usr/local/etc/rc.newwanip: IPv4 renewal is starting on 'ix1'   
2022-03-03T07:02:55-05:00   Error   opnsense   /usr/local/etc/rc.linkup: DEVD: Ethernet attached event for dynamic wan(ix1)   
2022-03-03T07:02:54-05:00   Error   opnsense   /usr/local/etc/rc.linkup: Clearing states for stale wan route on ix1   
2022-03-03T07:02:53-05:00   Critical   dhclient   exiting.   
2022-03-03T07:02:53-05:00   Error   dhclient   connection closed   

I can confirm that I did not experience this on previous versions, and rolling back to 22.1.1 gives me a stable system. This was all done using community release versions rather than development versions.

Current stable version is:
OPNsense 22.1.1_3-amd64
FreeBSD 13.0-STABLE
OpenSSL 1.1.1m 14 Dec 2021

Any ideas? I really appreciate any help.

Thank you,
Fox

Logged

peter.vynck

  • Newbie
  • *
  • Posts: 11
  • Karma: 1
    • View Profile
Re: WAN interface flapping with 22.1.2
« Reply #1 on: March 03, 2022, 03:44:34 pm »
Same problem here. I have 3 WAN-connections, all get their address from ISP through DHCP. The problematic interface is that one with a DHCP-delivered static address. When I disable that WAN-interface (MAC spoofing) the system becomes stable.
Did a fresh install on another machine with an older version, did the upgrades and face the exact same problem?!
Later today will try 2 things: revert to previous version on the original machine. And try a fresh installation with latest version on the other machine.
Logged

foxmanb

  • Newbie
  • *
  • Posts: 8
  • Karma: 0
    • View Profile
Re: WAN interface flapping with 22.1.2
« Reply #2 on: March 03, 2022, 05:20:33 pm »
I look forward to hearing what you find. It's a very strange issue.
Logged

subivoodoo

  • Newbie
  • *
  • Posts: 30
  • Karma: 1
    • View Profile
Re: WAN interface flapping with 22.1.2
« Reply #3 on: March 03, 2022, 05:29:47 pm »
Hi,

I bet you use intrusion detection in IPS mode together with MAC spoofing on your WAN?

I figured out the same issue here:
https://forum.opnsense.org/index.php?topic=26672.0

Workaround at the moment for me:
a) disable IPS or intrusion detection
or
b) remove the spoofed MAC from WAN
« Last Edit: March 03, 2022, 05:37:10 pm by subivoodoo »
Logged

foxmanb

  • Newbie
  • *
  • Posts: 8
  • Karma: 0
    • View Profile
Re: WAN interface flapping with 22.1.2
« Reply #4 on: March 03, 2022, 05:39:55 pm »
Hmm, I do use a spoofed mac, or at least at one time it was spoofed. I'm assuming I would need to generate my own MAC address rather than spoofing one? Xfinity requires a MAC be presented on the WAN interface.
Logged

subivoodoo

  • Newbie
  • *
  • Posts: 30
  • Karma: 1
    • View Profile
Re: WAN interface flapping with 22.1.2
« Reply #5 on: March 03, 2022, 05:47:01 pm »
During my tests it doesn't matter if it's a real MAC or not. As soon as there is some MAC entered on the same interface as intrusion detection with IPS is enabled, the if down/up happens after suricata/netmap is fully loaded.
Logged

aimdev

  • Full Member
  • ***
  • Posts: 111
  • Karma: 4
    • View Profile
Re: WAN interface flapping with 22.1.2
« Reply #6 on: March 03, 2022, 06:05:01 pm »
I raised this in
https://forum.opnsense.org/index.php?topic=26657.msg128900#msg128900
January 31, 2022, 12:44:15 pm

I have had no issues since.
For operational reasons I have not re-enabled suricata.
This issue did not occur with 21.x
LAN & WAN interfaces are not virtual, and are intel 

em0@pci0:0:31:6:   class=0x020000 rev=0x21 hdr=0x00 vendor=0x8086 device=0x156f subvendor=0x8086 subdevice=0x0000
    vendor     = 'Intel Corporation'
    device     = 'Ethernet Connection I219-LM'
    class      = network
    subclass   = ethernet
igb0@pci0:2:0:0:   class=0x020000 rev=0x03 hdr=0x00 vendor=0x8086 device=0x1539 subvendor=0x8086 subdevice=0x0000
    vendor     = 'Intel Corporation'
    device     = 'I211 Gigabit Network Connection'
    class      = network
    subclass   = ethernet

MAC not spoofed
« Last Edit: March 03, 2022, 07:28:03 pm by aimdev »
Logged

peter.vynck

  • Newbie
  • *
  • Posts: 11
  • Karma: 1
    • View Profile
Re: WAN interface flapping with 22.1.2
« Reply #7 on: March 03, 2022, 07:21:05 pm »
Hi, reading the posts here I decided to remove the MAC-spoofing. As a result everything is back to normal...
I downloaded updated rules for Suricata and enabled it: everything stays normal.

Regarding the MAC-spoofing: the MAC-address I had in the config was actually the real physical MAC of the interface. Can that be a reason?
Logged

aimdev

  • Full Member
  • ***
  • Posts: 111
  • Karma: 4
    • View Profile
Re: WAN interface flapping with 22.1.2
« Reply #8 on: March 03, 2022, 07:27:16 pm »
I would suggest that you keep a lookout on the console to see if the WAN drops.
In addition to my post, mac address was not spoofed.
Logged

smeetst

  • Newbie
  • *
  • Posts: 1
  • Karma: 0
    • View Profile
Re: WAN interface flapping with 22.1.2
« Reply #9 on: March 04, 2022, 08:30:56 am »
I have the same issue, however I have to spoof the mac.

But if I revert freeradius:
opnsense-revert -r 22.1.1 os-freeradius

the problem is gone.
The rest of the stack is up to date.
Logged

peter.vynck

  • Newbie
  • *
  • Posts: 11
  • Karma: 1
    • View Profile
Re: WAN interface flapping with 22.1.2
« Reply #10 on: March 04, 2022, 08:41:53 am »
Hi, personally not using os-freeradius here.

As for my remark about spoofing the real MAC-address: when I tried on another machine the real MAC-address was different. Meaning that it is the spoofing itself that seems to trigger the problem.

I will try to replicate the issue this weekend by re-introducing spoofing 'just for fun'.
Logged

subivoodoo

  • Newbie
  • *
  • Posts: 30
  • Karma: 1
    • View Profile
Re: WAN interface flapping with 22.1.2
« Reply #11 on: March 04, 2022, 07:26:55 pm »
In my case I do not have freeradius installed... but I reverted OpnSense to 22.1.1_3 (opnsense-revert -r 22.1.1 opnsense) and until now it also works for me on my clean test install (MAC spoofing + intrusion detection with IPS mode on).
« Last Edit: March 04, 2022, 07:29:59 pm by subivoodoo »
Logged

peter.vynck

  • Newbie
  • *
  • Posts: 11
  • Karma: 1
    • View Profile
Re: WAN interface flapping with 22.1.2
« Reply #12 on: March 06, 2022, 01:53:01 pm »
I introduced MAC-spoofing again and immediately the problems start all-over.
What surprises me is that the responsiveness from the router-GUI on the LAN-side gets crippled as well making it hard to change the settings back to 'normal'.


The log looks like this after entering a MAC-address:

2022-03-06T13:35:07   Error   opnsense    /usr/local/etc/rc.newwanip: IPv4 renewal is starting on 'igb0'
2022-03-06T13:35:02   Error   opnsense    /usr/local/etc/rc.linkup: DEVD: Ethernet attached event for dynamic opt1(igb0)
2022-03-06T13:34:59   Error   opnsense    /usr/local/etc/rc.filter_configure: Ignore down inet6 gateways : WAN1FIXED_DHCP
2022-03-06T13:34:59   Error   opnsense    /usr/local/etc/rc.filter_configure: ROUTING: keeping current default gateway '213.118.192.1'
2022-03-06T13:34:59   Error   opnsense    /usr/local/etc/rc.filter_configure: Ignore down inet gateways : WAN1FIXED_DHCP
2022-03-06T13:34:59   Error   opnsense    /interfaces.php: The WAN1FIXED_DHCP IPv4 interface address is invalid, skipping.
2022-03-06T13:34:59   Error   opnsense    /interfaces.php: Choose to bind WAN1FIXED_DHCP on  since we could not find a proper match.
2022-03-06T13:34:59   Error   opnsense    /interfaces.php: Adding static route for monitor 8.8.8.8 via 213.118.152.1
2022-03-06T13:34:59   Error   opnsense    /interfaces.php: Removing static route for monitor 8.8.8.8 via 213.118.152.1
2022-03-06T13:34:59   Error   opnsense    /interfaces.php: Adding static route for monitor 8.8.4.4 via 213.118.192.1
2022-03-06T13:34:59   Error   opnsense    /interfaces.php: Removing static route for monitor 8.8.4.4 via 213.118.192.1
2022-03-06T13:34:58   Error   opnsense    /interfaces.php: ROUTING: keeping current default gateway '213.118.192.1'
2022-03-06T13:34:58   Error   opnsense    /interfaces.php: ROUTING: setting IPv4 default route to 213.118.192.1
2022-03-06T13:34:58   Error   opnsense    /interfaces.php: ROUTING: IPv4 default gateway set to opt4
2022-03-06T13:34:58   Error   opnsense    /interfaces.php: ROUTING: entering configure using defaults
2022-03-06T13:34:57   Error   opnsense    /usr/local/etc/rc.filter_configure: ROUTING: keeping current default gateway '213.118.192.1'
2022-03-06T13:34:57   Error   opnsense    /interfaces.php: ROUTING: skipping IPv4 default route
2022-03-06T13:34:57   Error   opnsense    /interfaces.php: ROUTING: IPv4 default gateway set to opt4
2022-03-06T13:34:57   Error   opnsense    /interfaces.php: ROUTING: entering configure using 'opt1'
2022-03-06T13:34:57   Error   opnsense    /interfaces.php: The command '/sbin/dhclient -c '/var/etc/dhclient_opt1.conf' -p '/var/run/dhclient.igb0.pid' 'igb0'' returned exit code '15', the output was 'igb0: no link ...'
2022-03-06T13:34:57   Error   opnsense    /usr/local/etc/rc.linkup: DEVD: Ethernet detached event for dynamic opt1(igb0)
2022-03-06T13:34:56   Error   opnsense    /interfaces.php: Clearing states for stale opt1 route on igb0
2022-03-06T13:34:56   Critical   dhclient    exiting.
2022-03-06T13:34:56   Error   dhclient    connection closed
Logged

RickNY

  • Jr. Member
  • **
  • Posts: 59
  • Karma: 8
    • View Profile
Re: WAN interface flapping with 22.1.2
« Reply #13 on: March 06, 2022, 03:45:42 pm »

Also experienced WAN issues with 22.1.2_1 here... I was using MAC spoofing on the WAN interface, but removed it for troubleshooting.  I am not using IPS on this device.  One of the issues I have is that if my upstream device (cable modem) reboots, the DHCP client does not appear to get an IP again once the link returns -- the GUI just displays "dhcp" -- I have to manually reboot the entire firewall in order to get an IP on the WAN interface again. 

Second -- on reboots, my Wireguard VPN fails and stays that way because the initialization for that takes place while the WAN is still broken -- and even when it returns, it doesn't attempt to fix itself.. So I have to manually disable Wireguard and re-enable it for it to work.

Logged

Daemotrix

  • Newbie
  • *
  • Posts: 15
  • Karma: 0
    • View Profile
Re: WAN interface flapping with 22.1.2
« Reply #14 on: March 06, 2022, 04:33:42 pm »
I have also the same problem as described upper, I'm using HP 600 G2 DM with Intel NIC and also I'm using MAC spoofing. Problems with flapping started after the update, I also tried a fresh install, but without results. With older rls It started to work normally so there must be some bug in this rls.
Logged

  • Print
Pages: [1] 2 3 ... 12
« previous next »
  • OPNsense Forum »
  • English Forums »
  • 22.1 Legacy Series »
  • WAN interface flapping with 22.1.2
 

OPNsense is an OSS project © Deciso B.V. 2015 - 2022 All rights reserved
  • SMF 2.0.19 | SMF © 2021, Simple Machines
    Privacy Policy
    | XHTML | RSS | WAP2