Rondom WAN drop out

Started by annoniempjuh, October 04, 2020, 11:12:25 AM

Previous topic - Next topic
i struggle for months to get this issue tackled, but i cant found it.
biggest issue: NO LOGBOOKS! cant found anything in it what resulting in a dropping WAN, no idea why.
but the solution that is always working is restart suricata, restarting the wan interface only doesn't always work.
it doens't come back online by itself, i have to manually restart suricata.
i did try a full reset and put back the config, reinstalling the system... no luck...

its only doing this while watching Youtube video's, hammering the network with steam/sabnzdb/Plex/ or other things, doesn't tricker the drop out.
i tried a couple of days with suricata disabled, problem is still here...

is there a way to get uplift the logbook registration? its also spamming it with the logbook with syslog-ng messages:
syslog-ng[9873] Destination timeout has elapsed, closing connection; fd='5'
it bloody annoying!!!

Can i edit the config.xml and remove some of it without making it corrupt? like the IDS rule, i want the reset suricata...

can confirm, editing the config file and remove some old code will reset suricata etc. after this try i did a full reset and put the edited config back, works fine, but WAN still drops offline. mostly while whatsing some youtube video on my kodi machine...

more info, because its not fixed.

My ISP is Ziggo (dutch) its using only IPV4 on DHCP, the modem is an UBEE 1318ZG in Bridge modus.
OPNsense version: 20.7.3
hardware:
AMD ryzen 3 2200g
ASRock Fatal1ty B450 Gaming-ITX/ac
Crucial Ballistix Sport LT BLS8G4D32AESBK
Intel PRO/1000 PT Dual Port Server Adapter (PCI-e 4x) (e1000)

WAN drops offline, dpinger is showing it. it wont come back automatically, have to do a manual restart of suricata (it work always!)
the isp modem logbooks are fine, nothing what can cause this problem.
Logbooks are showing NOTHING!!!!!!! very annoying...

hope someone has some tip/tricks to try...

at 08:11:02 WAN drops offline
LOGBOOKS:
General:

Date                                 Process                   Line
2020-10-20T08:13:12 syslog-ng[29923] Destination timeout has elapsed, closing connection; fd='31'
2020-10-20T08:13:07 syslog-ng[29923] Destination timeout has elapsed, closing connection; fd='33'
2020-10-20T08:13:07 syslog-ng[29923] Destination timeout has elapsed, closing connection; fd='32'
2020-10-20T08:13:03 syslog-ng[29923] Destination timeout has elapsed, closing connection; fd='25'
2020-10-20T08:12:12 opnsense[54388] plugins_configure hosts (execute task : unbound_hosts_generate())
2020-10-20T08:12:12 opnsense[54388] plugins_configure hosts (execute task : dnsmasq_hosts_generate())
2020-10-20T08:12:12 opnsense[54388] plugins_configure hosts ()
2020-10-20T08:12:12 opnsense[54388] /usr/local/etc/rc.newwanip: On (IP address: 10.0.37.1) (interface: Pihole[opt5]) (real interface: em1_vlan1337).
2020-10-20T08:12:12 opnsense[54388] /usr/local/etc/rc.newwanip: IPv4 renewal is starting on 'em1_vlan1337'
2020-10-20T08:12:12 opnsense[6369] /usr/local/etc/rc.linkup: Hotplug event detected for Pihole(opt5) but ignoring since interface is configured with static IP (10.0.37.1 ::)
2020-10-20T08:12:12 opnsense[60071] plugins_configure hosts (execute task : unbound_hosts_generate())
2020-10-20T08:12:12 opnsense[60071] plugins_configure hosts (execute task : dnsmasq_hosts_generate())
2020-10-20T08:12:12 opnsense[60071] plugins_configure hosts ()
2020-10-20T08:12:12 opnsense[60071] /usr/local/etc/rc.newwanip: On (IP address: 10.0.13.1) (interface: GuestLan[opt3]) (real interface: em1_vlan13).
2020-10-20T08:12:12 opnsense[60071] /usr/local/etc/rc.newwanip: IPv4 renewal is starting on 'em1_vlan13'
2020-10-20T08:12:12 opnsense[51800] /usr/local/etc/rc.linkup: Hotplug event detected for GuestLan(opt3) but ignoring since interface is configured with static IP (10.0.13.1 ::)
2020-10-20T08:12:12 opnsense[12331] plugins_configure hosts (execute task : unbound_hosts_generate())
2020-10-20T08:12:12 opnsense[12331] plugins_configure hosts (execute task : dnsmasq_hosts_generate())
2020-10-20T08:12:12 opnsense[12331] plugins_configure hosts ()
2020-10-20T08:12:12 opnsense[12331] /usr/local/etc/rc.newwanip: On (IP address: 10.0.12.1) (interface: IOTlan[opt2]) (real interface: em1_vlan12).
2020-10-20T08:12:12 opnsense[12331] /usr/local/etc/rc.newwanip: IPv4 renewal is starting on 'em1_vlan12'
2020-10-20T08:12:12 opnsense[38844] /usr/local/etc/rc.linkup: Hotplug event detected for IOTlan(opt2) but ignoring since interface is configured with static IP (10.0.12.1 ::)
2020-10-20T08:12:11 opnsense[18637] plugins_configure hosts (execute task : unbound_hosts_generate())
2020-10-20T08:12:11 opnsense[18637] plugins_configure hosts (execute task : dnsmasq_hosts_generate())
2020-10-20T08:12:11 opnsense[18637] plugins_configure hosts ()
2020-10-20T08:12:11 opnsense[18637] /usr/local/etc/rc.newwanip: On (IP address: 10.0.11.1) (interface: NVRlan[opt1]) (real interface: em1_vlan11).
2020-10-20T08:12:11 opnsense[18637] /usr/local/etc/rc.newwanip: IPv4 renewal is starting on 'em1_vlan11'
2020-10-20T08:12:11 opnsense[48008] /usr/local/etc/rc.linkup: Hotplug event detected for NVRlan(opt1) but ignoring since interface is configured with static IP (10.0.11.1 ::)
2020-10-20T08:12:11 opnsense[7364] plugins_configure hosts (execute task : unbound_hosts_generate())
2020-10-20T08:12:11 opnsense[7364] plugins_configure hosts (execute task : dnsmasq_hosts_generate())
2020-10-20T08:12:11 opnsense[7364] plugins_configure hosts ()
2020-10-20T08:12:11 opnsense[7364] /usr/local/etc/rc.newwanip: On (IP address: 10.0.3.1) (interface: LAN[lan]) (real interface: em1).
2020-10-20T08:12:11 opnsense[7364] /usr/local/etc/rc.newwanip: IPv4 renewal is starting on 'em1'
2020-10-20T08:12:11 opnsense[21211] /usr/local/etc/rc.linkup: Hotplug event detected for LAN(lan) but ignoring since interface is configured with static IP (10.0.3.1 ::)
2020-10-20T08:12:09 opnsense[56743] /usr/local/etc/rc.dyndns: Dynamic DNS: (Success) IP Address Updated Successfully!
2020-10-20T08:12:09 opnsense[56743] /usr/local/etc/rc.dyndns: Dynamic DNS: updating cache file /var/cache/dyndns_wan_myddomain.duckdns.org_0.cache: XX.xx.xx.XX
2020-10-20T08:12:09 opnsense[56743] /usr/local/etc/rc.dyndns: Dynamic DNS (myddomain.duckdns.org): XX.xx.xx.XX extracted
2020-10-20T08:12:09 opnsense[56743] /usr/local/etc/rc.dyndns: Dynamic DNS (myddomain.duckdns.org): Current Service: custom
2020-10-20T08:12:09 opnsense[56743] /usr/local/etc/rc.dyndns: Dynamic DNS (myddomain.duckdns.org): _checkStatus() starting.
2020-10-20T08:12:09 upsmon[80213] Communications with UPS ups@10.0.3.2 established
2020-10-20T08:12:08 opnsense[56743] /usr/local/etc/rc.dyndns: Dynamic DNS (myddomain.duckdns.org via Custom): _update() starting.
2020-10-20T08:12:08 opnsense[56743] /usr/local/etc/rc.dyndns: Dynamic DNS (myddomain.duckdns.orgg): running dyndns_failover_interface for wan. found em0
2020-10-20T08:12:08 opnsense[56743] /usr/local/etc/rc.dyndns: Dynamic DNS (myddomain.duckdns.org): XX.xx.xx.XX extracted
2020-10-20T08:12:08 opnsense[56743] /usr/local/etc/rc.dyndns: Dynamic DNS: updatedns() starting
2020-10-20T08:12:07 opnsense[28380] plugins_configure dns (execute task : unbound_configure_do())
2020-10-20T08:12:07 opnsense[28380] plugins_configure dns (execute task : dnsmasq_configure_do())
2020-10-20T08:12:07 opnsense[28380] plugins_configure dns ()
2020-10-20T08:12:07 opnsense[28380] plugins_configure dhcp (execute task : dhcpd_dhcp_configure())
2020-10-20T08:12:07 opnsense[28380] plugins_configure dhcp ()
2020-10-20T08:12:07 opnsense[28380] plugins_configure ipsec (execute task : ipsec_configure_do(,wan))
2020-10-20T08:12:07 opnsense[28380] plugins_configure ipsec (,wan)
2020-10-20T08:12:07 opnsense[28380] /usr/local/etc/rc.linkup: ROUTING: keeping current default gateway 'XX.xx.xx.XX'
2020-10-20T08:12:07 opnsense[28380] /usr/local/etc/rc.linkup: ROUTING: setting IPv4 default route to XX.xx.xx.XX
2020-10-20T08:12:07 opnsense[28380] /usr/local/etc/rc.linkup: ROUTING: IPv4 default gateway set to wan
2020-10-20T08:12:07 opnsense[28380] /usr/local/etc/rc.linkup: ROUTING: entering configure using 'wan'
2020-10-20T08:12:07 opnsense[98376] plugins_configure newwanip (execute task : webgui_configure_do(,wan))
2020-10-20T08:12:07 opnsense[98376] plugins_configure newwanip (execute task : vxlan_configure_interface())
2020-10-20T08:12:07 opnsense[98376] plugins_configure newwanip (execute task : unbound_configure_do(,wan))
2020-10-20T08:12:07 opnsense[98376] plugins_configure newwanip (execute task : openssh_configure_do(,wan))
2020-10-20T08:12:07 opnsense[98376] plugins_configure newwanip (execute task : opendns_configure_do())
2020-10-20T08:12:07 opnsense[98376] plugins_configure newwanip (execute task : ntpd_configure_defer())
2020-10-20T08:12:06 opnsense[98376] /usr/local/etc/rc.newwanip: Dynamic DNS: (Success) IP Address Updated Successfully!
2020-10-20T08:12:06 opnsense[98376] /usr/local/etc/rc.newwanip: Dynamic DNS: updating cache file /var/cache/dyndns_wan_myddomain.duckdns.org_0.cache: XX.xx.xx.XX
2020-10-20T08:12:06 opnsense[98376] /usr/local/etc/rc.newwanip: Dynamic DNS (myddomain.duckdns.org): XX.xx.xx.XX extracted
2020-10-20T08:12:06 opnsense[98376] /usr/local/etc/rc.newwanip: Dynamic DNS (myddomain.duckdns.org): Current Service: custom
2020-10-20T08:12:06 opnsense[98376] /usr/local/etc/rc.newwanip: Dynamic DNS (myddomain.duckdns.org): _checkStatus() starting.
2020-10-20T08:12:04 upsmon[80213] Communications with UPS ups@10.0.3.2 lost
2020-10-20T08:12:04 upsmon[80213] Poll UPS [ups@10.0.3.2] failed - Server disconnected
2020-10-20T08:12:04 opnsense[98376] /usr/local/etc/rc.newwanip: Dynamic DNS (myddomain.duckdns.org via Custom): _update() starting.
2020-10-20T08:12:04 opnsense[98376] /usr/local/etc/rc.newwanip: Dynamic DNS (myddomain.duckdns.org): running dyndns_failover_interface for wan. found em0
2020-10-20T08:12:04 opnsense[98376] /usr/local/etc/rc.newwanip: Dynamic DNS (myddomain.duckdns.org): XX.xx.xx.XX extracted
2020-10-20T08:12:04 opnsense[98376] /usr/local/etc/rc.newwanip: Dynamic DNS: updatedns() starting
2020-10-20T08:12:04 opnsense[98376] plugins_configure newwanip (execute task : dyndns_configure_do(,wan))
2020-10-20T08:12:04 opnsense[98376] plugins_configure newwanip (,wan)
2020-10-20T08:12:04 opnsense[98376] /usr/local/etc/rc.newwanip: Resyncing OpenVPN instances for interface WAN.
2020-10-20T08:12:04 opnsense[98376] plugins_configure vpn (execute task : openvpn_configure_do(,wan))
2020-10-20T08:12:04 opnsense[98376] plugins_configure vpn (execute task : ipsec_configure_do(,wan))
2020-10-20T08:12:04 opnsense[98376] plugins_configure vpn (,wan)
2020-10-20T08:12:03 opnsense[98376] plugins_configure monitor (execute task : dpinger_configure_do())
2020-10-20T08:12:03 opnsense[98376] plugins_configure monitor ()
2020-10-20T08:12:03 opnsense[98376] /usr/local/etc/rc.newwanip: ROUTING: keeping current default gateway 'XX.xx.xx.XX'
2020-10-20T08:12:03 opnsense[98376] /usr/local/etc/rc.newwanip: ROUTING: setting IPv4 default route to XX.xx.xx.XX
2020-10-20T08:12:03 opnsense[98376] /usr/local/etc/rc.newwanip: ROUTING: IPv4 default gateway set to wan
2020-10-20T08:12:03 opnsense[98376] /usr/local/etc/rc.newwanip: ROUTING: entering configure using 'wan'
2020-10-20T08:12:03 opnsense[98376] plugins_configure hosts (execute task : unbound_hosts_generate())
2020-10-20T08:12:03 opnsense[98376] plugins_configure hosts (execute task : dnsmasq_hosts_generate())
2020-10-20T08:12:03 opnsense[98376] plugins_configure hosts ()
2020-10-20T08:12:03 opnsense[98376] /usr/local/etc/rc.newwanip: On (IP address: XX.xx.xx.XX) (interface: WAN[wan]) (real interface: em0).
2020-10-20T08:12:03 opnsense[98376] /usr/local/etc/rc.newwanip: IPv4 renewal is starting on 'em0'
2020-10-20T08:12:03 dhclient[28531] Creating resolv.conf
2020-10-20T08:12:03 dhclient[58696] route add default XX.xx.xx.XX
2020-10-20T08:12:03 dhclient[49439] New Routers (em0): XX.xx.xx.XX
2020-10-20T08:12:03 dhclient[69234] New Broadcast Address (em0): 255.255.255.255
2020-10-20T08:12:03 dhclient[91537] New Subnet Mask (em0): 255.255.255.0
2020-10-20T08:12:03 dhclient[15365] New IP Address (em0): XX.xx.xx.XX
2020-10-20T08:12:03 dhclient[80651] Comparing IPs: Old: "same as new" New: XX.xx.xx.XX
2020-10-20T08:12:03 dhclient[44443] Starting delete_old_states()
2020-10-20T08:12:03 dhclient[92185] Removing states from old IP 'XX.xx.xx.XX' (new IP '')
2020-10-20T08:12:03 dhclient[54838] Comparing IPs: Old: XX.xx.xx.XX New:
2020-10-20T08:12:03 dhclient[44874] Starting delete_old_states()
2020-10-20T08:12:03 opnsense[28380] /usr/local/etc/rc.linkup: HOTPLUG: Configuring interface wan
2020-10-20T08:12:03 opnsense[28380] /usr/local/etc/rc.linkup: DEVD Ethernet attached event for wan
2020-10-20T08:12:02 opnsense[63404] /usr/local/etc/rc.linkup: Hotplug event detected for Pihole(opt5) but ignoring since interface is configured with static IP (10.0.37.1 ::)
2020-10-20T08:12:02 opnsense[45932] /usr/local/etc/rc.linkup: Hotplug event detected for GuestLan(opt3) but ignoring since interface is configured with static IP (10.0.13.1 ::)
2020-10-20T08:12:02 opnsense[70505] /usr/local/etc/rc.linkup: Hotplug event detected for IOTlan(opt2) but ignoring since interface is configured with static IP (10.0.12.1 ::)
2020-10-20T08:12:01 opnsense[58830] /usr/local/etc/rc.linkup: Hotplug event detected for NVRlan(opt1) but ignoring since interface is configured with static IP (10.0.11.1 ::)
2020-10-20T08:12:01 opnsense[28598] /usr/local/etc/rc.linkup: Hotplug event detected for LAN(lan) but ignoring since interface is configured with static IP (10.0.3.1 ::)
2020-10-20T08:12:00 opnsense[64257] /usr/local/etc/rc.dyndns: Curl error occurred: Resolving timed out after 15003 milliseconds
2020-10-20T08:12:00 opnsense[64257] /usr/local/etc/rc.dyndns: Dynamic DNS (myddomain.duckdns.org): Current Service: custom
2020-10-20T08:12:00 opnsense[64257] /usr/local/etc/rc.dyndns: Dynamic DNS (myddomain.duckdns.org): _checkStatus() starting.
2020-10-20T08:11:59 opnsense[40532] /usr/local/etc/rc.linkup: Clearing states for stale wan route on em0
2020-10-20T08:11:59 dhclient[66399] exiting.
2020-10-20T08:11:59 dhclient[66399] connection closed
2020-10-20T08:11:59 opnsense[40532] /usr/local/etc/rc.linkup: DEVD Ethernet detached event for wan
2020-10-20T08:11:52 webgui[11298] /index.php: Successful login for user 'root' from: 10.0.3.40
2020-10-20T08:11:02 opnsense[64257] /usr/local/etc/rc.dyndns: Dynamic DNS (myddomain.duckdns.org via Custom): _update() starting.
2020-10-20T08:11:02 opnsense[64257] /usr/local/etc/rc.dyndns: Dynamic DNS (myddomain.duckdns.org): running dyndns_failover_interface for wan. found em0
2020-10-20T08:11:02 opnsense[64257] /usr/local/etc/rc.dyndns: Dynamic DNS (myddomain.duckdns.org): XX.xx.xx.XX extracted
2020-10-20T08:11:02 opnsense[64257] /usr/local/etc/rc.dyndns: Dynamic DNS: updatedns() starting
2020-10-20T08:10:50 syslog-ng[29923] Destination timeout has elapsed, closing connection; fd='5'
2020-10-20T08:09:50 syslog-ng[29923] Destination timeout has elapsed, closing connection; fd='26'
2020-10-20T08:09:46 syslog-ng[29923] Destination timeout has elapsed, closing connection; fd='5'
2020-10-20T08:08:46 syslog-ng[29923] Destination timeout has elapsed, closing connection; fd='5'
2020-10-20T08:07:46 syslog-ng[29923] Destination timeout has elapsed, closing connection; fd='5'
2020-10-20T08:06:46 syslog-ng[29923] Destination timeout has elapsed, closing connection; fd='5'

at the moment WAN drops offline, DynDNS is started but will fail...

dpinger:
2020-10-20T08:13:39 dpinger[47604] send_interval 1000ms loss_interval 2000ms time_period 60000ms report_interval 0ms data_len 0 alert_interval 1000ms latency_alarm 500ms loss_alarm 20% dest_addr xx.xx.xx.xx bind_addr xx.xx.xx.xx identifier "WAN "
2020-10-20T08:13:39 dpinger[36894] WAN xx.xx.xx.xx: sendto error: 65
2020-10-20T08:13:38 dpinger[36894] WAN xx.xx.xx.xx: sendto error: 65
2020-10-20T08:13:37 dpinger[36894] WAN xx.xx.xx.xx: sendto error: 65
2020-10-20T08:13:36 dpinger[36894] WAN xx.xx.xx.xx: sendto error: 65
2020-10-20T08:13:35 dpinger[36894] WAN xx.xx.xx.xx: sendto error: 65
2020-10-20T08:13:34 dpinger[36894] WAN xx.xx.xx.xx: sendto error: 65
2020-10-20T08:13:33 dpinger[36894] WAN xx.xx.xx.xx: sendto error: 65
2020-10-20T08:13:32 dpinger[36894] WAN xx.xx.xx.xx: sendto error: 65
2020-10-20T08:13:31 dpinger[36894] WAN xx.xx.xx.xx: sendto error: 65
2020-10-20T08:13:30 dpinger[36894] WAN xx.xx.xx.xx: sendto error: 50
2020-10-20T08:12:03 dpinger[36894] send_interval 1000ms loss_interval 2000ms time_period 60000ms report_interval 0ms data_len 0 alert_interval 1000ms latency_alarm 500ms loss_alarm 20% dest_addr xx.xx.xx.xx bind_addr xx.xx.xx.xx identifier "WAN "
2020-10-20T08:12:02 dpinger[43866] WAN xx.xx.xx.xx: sendto error: 65
2020-10-20T08:12:01 dpinger[43866] WAN xx.xx.xx.xx: sendto error: 65
2020-10-20T08:12:00 dpinger[43866] WAN xx.xx.xx.xx: sendto error: 65
2020-10-20T08:11:59 dpinger[43866] WAN xx.xx.xx.xx: sendto error: 50
2020-10-20T08:11:01 dpinger[4152] GATEWAY ALARM: WAN (Addr: xx.xx.xx.xx Alarm: 1 RTT: 12038ms RTTd: 8903ms Loss: 22%)
2020-10-20T08:11:01 dpinger[43866] WAN xx.xx.xx.xx: Alarm latency 12038us stddev 8903us loss 22%
2020-10-17T22:43:10 dpinger[43290] GATEWAY ALARM: WAN (Addr: xx.xx.xx.xx Alarm: 0 RTT: 9130ms RTTd: 1408ms Loss: 0%)

experiencing possibly the same issue with 20.7.5
...mostly during MS Teams sessions.