Flapping interfaces makes connection instable!

[gh0st]

This is a behaviour OPNsense has started to do the latest week...

Code Select Expand

Nov 4 20:32:32 opnsense: /usr/local/etc/rc.newwanip: On (IP address: 10.128.242.23) (interface: OPT1[opt1]) (real interface: ovpnc1).
Nov 4 20:32:32 opnsense: /usr/local/etc/rc.newwanip: IP renewal is starting on 'ovpnc1'
Nov 4 20:32:32 configd.py: [bce5f636-c7ba-4fd4-b1e8-e90b5387a4a2] rc.newwanip starting ovpnc1
Nov 4 20:32:32 kernel: ovpnc1: link state changed to UP
Nov 4 20:32:30 configd.py: [5e3ce8de-6d8c-4103-aa20-b1321fa8c936] Reloading filter
Nov 4 20:32:30 kernel: ovpnc1: link state changed to DOWN
Nov 4 20:03:48 opnsense: /usr/local/etc/rc.newwanip: On (IP address: 10.128.80.11) (interface: OPT1[opt1]) (real interface: ovpnc1).
Nov 4 20:03:48 opnsense: /usr/local/etc/rc.newwanip: IP renewal is starting on 'ovpnc1'
Nov 4 20:03:48 configd.py: [b2973947-dea0-474e-9de1-1ee4f6d5e78d] rc.newwanip starting ovpnc1
Nov 4 20:03:48 kernel: ovpnc1: link state changed to UP
Nov 4 20:03:46 configd.py: [e6cee557-fa5b-47b7-a853-a8e879f22b06] Reloading filter
Nov 4 20:03:46 kernel: ovpnc1: link state changed to DOWN
Nov 4 19:24:34 opnsense: /usr/local/etc/rc.newwanip: On (IP address: 10.128.194.8) (interface: OPT1[opt1]) (real interface: ovpnc1).
Nov 4 19:24:34 opnsense: /usr/local/etc/rc.newwanip: IP renewal is starting on 'ovpnc1'
Nov 4 19:24:34 configd.py: [304bcb06-9621-4cf5-9957-4b5963a56942] rc.newwanip starting ovpnc1
Nov 4 19:24:34 kernel: ovpnc1: link state changed to UP
Nov 4 19:24:32 configd.py: [4c548643-c6f5-43b9-8100-23acbed4a365] Reloading filter
Nov 4 19:24:32 kernel: ovpnc1: link state changed to DOWN

I'm running the 17.7, and this behaviour was also on the 17.4 before the update.

[mimugmail]

What is the exact problem here? When an Interface flaps it's always unstable ..

[Oxygen61]

Hi gh0st,

Last week you say? What is your ISP? Just asking since i have the same issues. ;)
What exactly is the OpenVPN Log File saying about your Interface flapping? Any bigger errors/issues?

Best regards,
Oxy

[gh0st]

Hi gh0st,

Last week you say? What is your ISP? Just asking since i have the same issues. ;)
What exactly is the OpenVPN Log File saying about your Interface flapping? Any bigger errors/issues?

Best regards,
Oxy

Hi Oxy!

Living in Sweden and are using Tele 2 as ISP.
Nothing in the OpenVPN log file, but more (the above) in the OPNsense log file.

And one thing I also noticed is that when the interface begin to flap, I lost control to one of my servers outside with ERR_REFUSE_TO_CONNECT. And once I disable the interface for OVPN it starts work again, and also after I've restarted the interface it works until it flaps again.

[NilsS]

Do you use suricata with IPS mode?

[gh0st]

And here we have the log file from OpenVPN:

Code Select Expand

Nov 12 00:27:17
configd.py: unable to sendback response [OK ] for [interface][linkup][['start', 'bge0']] {8ab20306-83ee-4a53-bff0-252045b13d71}, message was Traceback (most recent call last): File "/usr/local/opnsense/service/modules/processhandler.py", line 202, in run self.connection.sendall('%s\n' % result) File "/usr/local/lib/python2.7/socket.py", line 228, in meth return getattr(self._sock,name)(*args) error: [Errno 32] Broken pipe
Nov 12 00:27:15
opnsense: /usr/local/etc/rc.linkup: Accept router advertisements on interface bge0
Nov 12 00:27:15
opnsense: /usr/local/etc/rc.linkup: The command '/sbin/dhclient -c /var/etc/dhclient_wan.conf bge0 > /tmp/bge0_output 2> /tmp/bge0_error_output' returned exit code '1', the output was ''
Nov 12 00:27:15
opnsense: /usr/local/etc/rc.linkup: HOTPLUG: Configuring interface wan
Nov 12 00:27:15
opnsense: /usr/local/etc/rc.linkup: DEVD Ethernet attached event for wan
Nov 12 00:27:15
configd.py: [48068672-d9dc-425c-81d6-5efb311b5e93] Linkup starting bge0
Nov 12 00:27:15
opnsense: /usr/local/etc/rc.linkup: Clearing states to old gateway 192.168.1.1.
Nov 12 00:27:15
opnsense: /usr/local/etc/rc.linkup: The command '/sbin/route add -'inet' default '192.168.1.1'' returned exit code '1', the output was 'route: writing to routing socket: Network is unreachable add net default: gateway 192.168.1.1 fib 0: Network is unreachable'
Nov 12 00:27:15
opnsense: /usr/local/etc/rc.linkup: ROUTING: setting IPv4 default route to 192.168.1.1
Nov 12 00:27:15
opnsense: /usr/local/etc/rc.linkup: Accept router advertisements on interface bge0
Nov 12 00:27:15
opnsense: /usr/local/etc/rc.linkup: The command '/sbin/dhclient -c /var/etc/dhclient_wan.conf bge0 > /tmp/bge0_output 2> /tmp/bge0_error_output' returned exit code '15', the output was ''
Nov 12 00:27:15
opnsense: /usr/local/etc/rc.linkup: DEVD Ethernet detached event for wan
Nov 12 00:27:15
configd.py: [9fdb2d99-f4b8-45cd-a445-c2a955147a9a] Linkup stopping bge0
Nov 12 00:27:15
configd_ctl.py: error in configd communication Traceback (most recent call last): File "/usr/local/opnsense/service/configd_ctl.py", line 65, in exec_config_cmd line = sock.recv(65536) timeout: timed out
Nov 12 00:26:07
kernel: arpresolve: can't allocate llinfo for XX.XX.XX.XX on bge0
Nov 12 00:26:02
kernel: bge0: link state changed to UP
Nov 12 00:25:45
kernel: bge0: link state changed to DOWN
Nov 12 00:25:35
kernel: arpresolve: can't allocate llinfo for XX.XX.XX.XX on bge0
Nov 12 00:25:20
opnsense: /usr/local/etc/rc.newwanip: Resyncing OpenVPN instances for interface WAN.
Nov 12 00:25:20
opnsense: /usr/local/etc/rc.newwanip: ROUTING: setting IPv4 default route to 192.168.1.1
Nov 12 00:25:20
opnsense: /usr/local/etc/rc.newwanip: On (IP address: 192.168.1.33) (interface: WAN[wan]) (real interface: bge0).
Nov 12 00:25:20
opnsense: /usr/local/etc/rc.newwanip: IP renewal is starting on 'bge0'
Nov 12 00:25:15
opnsense: /usr/local/etc/rc.linkup: HOTPLUG: Configuring interface wan
Nov 12 00:25:15
opnsense: /usr/local/etc/rc.linkup: DEVD Ethernet attached event for wan
Nov 12 00:25:15
configd.py: [8ab20306-83ee-4a53-bff0-252045b13d71] Linkup starting bge0
Nov 12 00:25:15
kernel: bge0: link state changed to UP
Nov 12 00:25:09
opnsense: /usr/local/etc/rc.linkup: Clearing states to old gateway XX.XX.XX.
Nov 12 00:25:07
configd.py: [26c3a8f1-c75d-4388-97cd-a06edd977ad1] Reloading filter
Nov 12 00:25:06
configd.py: [4e8bb01c-2f28-4b9e-8229-ca6d12260e4e] Reloading filter
Nov 12 00:25:06
kernel: ovpnc1: link state changed to DOWN
Nov 12 00:25:05
kernel: arpresolve: can't allocate llinfo for XX.XX.XX.XX on bge0
Nov 12 00:25:04
kernel: arpresolve: can't allocate llinfo for XX.XX.XX.XX on bge0
Nov 12 00:25:04
kernel: arpresolve: can't allocate llinfo for XX.XX.XX.XX on bge0
Nov 12 00:25:03
kernel: arpresolve: can't allocate llinfo for XX.XX.XX.XX on bge0
Nov 12 00:25:02
opnsense: /usr/local/etc/rc.newwanipv6: Resyncing OpenVPN instances for interface WAN.
Nov 12 00:25:02
opnsense: /usr/local/etc/rc.newwanipv6: ROUTING: setting IPv6 default route to fe80::402:d8ff:fedf:1285
Nov 12 00:25:02
opnsense: /usr/local/etc/rc.newwanipv6: The command '/sbin/route add -'inet' default 'XX.XX.XX.XX'' returned exit code '1', the output was 'route: writing to routing socket: Network is unreachable add net default: gateway 37.2.112.244 fib 0: Network is unreachable'
Nov 12 00:25:02
opnsense: /usr/local/etc/rc.newwanipv6: ROUTING: setting IPv4 default route to XX.XX.XX.XX
Nov 12 00:25:02
opnsense: /usr/local/etc/rc.newwanipv6: On (IP address: 2a00:801:2d0:44c5:1e98:ecff:fe0f:a80c) (interface: WAN[wan]) (real interface: bge0).
Nov 12 00:25:02
opnsense: /usr/local/etc/rc.newwanipv6: IP renewal is starting on 'bge0'
Nov 12 00:25:02
configd.py: [ff904c45-020b-4b06-90de-da83187ffdcf] rc.newwanip starting bge0
Nov 12 00:25:02
opnsense: /usr/local/etc/rc.linkup: DEVD Ethernet detached event for wan
Nov 12 00:25:02
configd.py: [8c836d4f-6842-4f7a-91f8-2f8a3b4b02ea] Linkup stopping bge0
Nov 12 00:25:02
kernel: bge0: link state changed to DOWN


[gh0st]

Do you use suricata with IPS mode?

Nope!