os-openconnect / all traffic stops when server forces disconnect

[haukened]

I already opened this as a github issue, but perhaps you guys have some insight:
https://github.com/opnsense/plugins/issues/1692

Here's the summary:

**Describe the bug**
Recently installed os-openconnect to connect to Cisco ASA firewall as a VPN Client. Configuration was working with associated NAT statement allowing LAN to PAT through the ocvpn interface to the company.  After a certain amount of idle time, the server forces disconnect and OPNsense stops responding to traffic on all interfaces until the LAN/WAN are physically UP/DOWN (by unplugging) since the firewall UI/SSH are no longer available.

**To Reproduce**
Steps to reproduce the behavior:

• Install os-openconnect
• Configure with server, username, password
• Start OpenConnect service
• Configure NAT statement to NAT LAN to OpenConnect interface IP
• Test working connection using ocvpn interface
• Wait some amount of time (varies based on server config)
• Server forces idle disconnect
• OPNsense stops responding to traffic on ALL interfaces.
• Physically up/down LAN and WAN interface
• Connectivity is restored.
• Log into OPNsense admin page
• Observe OpenConnect service is stopped.

**Expected behavior**

• Server forces disconnect
• ocvpn interface goes down
• all other interfaces continue to work as normal

Firewall should treat it as any other downed interface.

**Relevant log files**

Code: [Select]

2020-02-10T13:01:18 dhcp6c[62879]: Sending Solicit
2020-02-10T12:59:30 dhcp6c[62879]: Sending Solicit
2020-02-10T12:58:22 dhcp6c[62879]: Sending Solicit
2020-02-10T12:57:48 dhcp6c[62879]: Sending Solicit
2020-02-10T12:57:30 dhcp6c[62879]: Sending Solicit
## internet is now working again  ##
2020-02-10T12:57:28 opnsense: plugins_configure newwanip (execute task : webgui_configure_do(,wan))
2020-02-10T12:57:28 opnsense: plugins_configure newwanip (execute task : vxlan_configure_interface())
2020-02-10T12:57:26 opnsense: plugins_configure newwanip (execute task : unbound_configure_do(,wan))
2020-02-10T12:57:26 opnsense: plugins_configure newwanip (execute task : openssh_configure_do(,wan))
2020-02-10T12:57:26 opnsense: plugins_configure newwanip (execute task : opendns_configure_do())
2020-02-10T12:57:26 opnsense: plugins_configure newwanip (execute task : ntpd_configure_defer())
entry.
2020-02-10T12:57:22 opnsense: /usr/local/etc/rc.newwanip: Dynamic DNS: (Success) IP address updated successfully (XXXXXXXXXX)
2020-02-10T12:57:22 opnsense: /usr/local/etc/rc.newwanip: Dynamic DNS: updating cache file /var/cache/dyndns_wan_XXXXXXXXX_1.cache: XXXXXXXXX
2020-02-10T12:57:22 dhcp6c[62879]: Sending Solicit
2020-02-10T12:57:20 opnsense: /usr/local/etc/rc.dyndns: Dynamic DNS (XXXXXXXXX): No change in my IP address and/or 25 days has not passed. Not updating dynamic DNS entry.
2020-02-10T12:57:20 opnsense: /usr/local/etc/rc.newwanip: Dynamic DNS: (Success) IP address updated successfully (XXXXXXXXXXX)
2020-02-10T12:57:20 opnsense: /usr/local/etc/rc.newwanip: Dynamic DNS: updating cache file /var/cache/dyndns_XXXXXXXXXXX_0.cache: XXXXXX
2020-02-10T12:57:17 dhcp6c[62879]: Sending Solicit
2020-02-10T12:57:15 dhcp6c[62879]: Sending Solicit
2020-02-10T12:57:14 opnsense: plugins_configure newwanip (execute task : dyndns_configure_do(,wan))
2020-02-10T12:57:14 opnsense: plugins_configure newwanip (,wan)
2020-02-10T12:57:14 opnsense: /usr/local/etc/rc.newwanip: Resyncing OpenVPN instances for interface WAN.
2020-02-10T12:57:14 opnsense: plugins_configure vpn (execute task : openvpn_configure_do(,wan))
2020-02-10T12:57:14 opnsense: plugins_configure vpn (execute task : ipsec_configure_do(,wan))
2020-02-10T12:57:14 kernel: pflog0: promiscuous mode enabled
2020-02-10T12:57:14 opnsense: plugins_configure vpn (,wan)
2020-02-10T12:57:14 kernel: pflog0: promiscuous mode disabled
2020-02-10T12:57:14 dhcp6c[62879]: Sending Solicit
2020-02-10T12:57:14 opnsense: /usr/local/etc/rc.newwanip: The WAN_PPPOE monitor address is empty, skipping.
2020-02-10T12:57:14 opnsense: /usr/local/etc/rc.newwanip: The WAN_DHCP6 monitor address is empty, skipping.
2020-02-10T12:57:14 opnsense: plugins_configure monitor (execute task : dpinger_configure_do())
2020-02-10T12:57:14 opnsense: plugins_configure monitor ()
2020-02-10T12:57:14 opnsense: /usr/local/etc/rc.newwanip: ROUTING: skipping IPv6 default route
2020-02-10T12:57:14 opnsense: /usr/local/etc/rc.newwanip: ROUTING: IPv6 default gateway set to wan
2020-02-10T12:57:14 opnsense: /usr/local/etc/rc.newwanip: ROUTING: keeping current default gateway 'XXXXXXXX'
2020-02-10T12:57:14 opnsense: /usr/local/etc/rc.newwanip: ROUTING: setting IPv4 default route to XXXXXXXXX
2020-02-10T12:57:14 opnsense: /usr/local/etc/rc.newwanip: ROUTING: IPv4 default gateway set to wan
2020-02-10T12:57:14 opnsense: /usr/local/etc/rc.newwanip: ROUTING: entering configure using 'wan'
2020-02-10T12:57:14 opnsense: plugins_configure hosts (execute task : unbound_hosts_generate())
2020-02-10T12:57:14 opnsense: plugins_configure hosts (execute task : dnsmasq_hosts_generate())
2020-02-10T12:57:14 opnsense: plugins_configure hosts ()
2020-02-10T12:57:14 opnsense: /usr/local/etc/rc.newwanip: On (IP address: XXXXXXXXX) (interface: WAN[wan]) (real interface: pppoe0).
2020-02-10T12:57:14 opnsense: /usr/local/etc/rc.newwanip: IP renewal is starting on 'pppoe0'
2020-02-10T12:57:14 opnsense: plugins_configure dns (execute task : unbound_configure_do())
2020-02-10T12:57:14 opnsense: plugins_configure dns (execute task : dnsmasq_configure_do())
2020-02-10T12:57:14 opnsense: plugins_configure dns ()
2020-02-10T12:57:14 opnsense: /usr/local/etc/rc.linkup: Warning! dhcpd_radvd_configure(auto) found no suitable IPv6 address on bce3
2020-02-10T12:57:14 opnsense: plugins_configure dhcp (execute task : dhcpd_dhcp_configure())
2020-02-10T12:57:14 opnsense: plugins_configure dhcp ()
2020-02-10T12:57:14 opnsense: plugins_configure ipsec (execute task : ipsec_configure_do(,wan))
2020-02-10T12:57:14 opnsense: plugins_configure ipsec (,wan)
2020-02-10T12:57:14 opnsense: /usr/local/etc/rc.linkup: ROUTING: skipping IPv6 default route
2020-02-10T12:57:14 opnsense: /usr/local/etc/rc.linkup: ROUTING: IPv6 default gateway set to wan
2020-02-10T12:57:14 opnsense: /usr/local/etc/rc.linkup: ROUTING: entering configure using 'wan'
2020-02-10T12:57:14 dhcp6c[62879]: restarting
2020-02-10T12:57:14 dhcp6c: RTSOLD script - Sending SIGHUP to dhcp6c for interface wan(bce2_vlan201)
2020-02-10T12:57:14 opnsense: /usr/local/etc/rc.linkup: Accept router advertisements on interface bce2_vlan201
2020-02-10T12:57:14 kernel: ng0: changing name to 'pppoe0'
2020-02-10T12:57:14 opnsense: /usr/local/etc/rc.linkup: The command '/sbin/ifconfig 'pppoe0' inet6 -accept_rtadv' returned exit code '1', the output was 'ifconfig: interface pppoe0 does not exist'
2020-02-10T12:57:13 opnsense: /usr/local/etc/rc.linkup: HOTPLUG: Configuring interface wan
2020-02-10T12:57:13 opnsense: /usr/local/etc/rc.linkup: DEVD Ethernet attached event for wan
2020-02-10T12:57:11 opnsense: /usr/local/etc/rc.linkup: DEVD Ethernet detached event for wan
2020-02-10T12:57:11 dhcp6c[62879]: Sending Solicit
2020-02-10T12:57:09 kernel: bce2_vlan201: link state changed to UP
2020-02-10T12:57:09 kernel: bce2: link state changed to UP
2020-02-10T12:57:09 kernel: bce2: Gigabit link up!
2020-02-10T12:57:06 dhcp6c[62879]: Sending Solicit
2020-02-10T12:57:04 dhcp6c[62879]: Sending Solicit
2020-02-10T12:57:03 kernel: bce2_vlan201: link state changed to DOWN
2020-02-10T12:57:03 kernel: bce2: link state changed to DOWN
2020-02-10T12:57:03 dhcp6c[62879]: Sending Solicit
## LAN interface is now accessible ##
2020-02-10T12:57:03 opnsense: plugins_configure dns (execute task : unbound_configure_do())
2020-02-10T12:57:03 opnsense: plugins_configure dns (execute task : dnsmasq_configure_do())
2020-02-10T12:57:03 opnsense: plugins_configure dns ()
2020-02-10T12:57:03 opnsense: /usr/local/etc/rc.linkup: Warning! dhcpd_radvd_configure(auto) found no suitable IPv6 address on bce3
2020-02-10T12:57:03 opnsense: plugins_configure dhcp (execute task : dhcpd_dhcp_configure())
2020-02-10T12:57:03 opnsense: plugins_configure dhcp ()
2020-02-10T12:57:03 opnsense: plugins_configure ipsec (execute task : ipsec_configure_do(,lan))
2020-02-10T12:57:03 opnsense: plugins_configure ipsec (,lan)
2020-02-10T12:57:03 opnsense: /usr/local/etc/rc.linkup: ROUTING: skipping IPv6 default route
2020-02-10T12:57:03 opnsense: /usr/local/etc/rc.linkup: ROUTING: IPv6 default gateway set to wan
2020-02-10T12:57:03 opnsense: /usr/local/etc/rc.linkup: ROUTING: skipping IPv4 default route
2020-02-10T12:57:03 opnsense: /usr/local/etc/rc.linkup: ROUTING: IPv4 default gateway set to wan
2020-02-10T12:57:03 opnsense: /usr/local/etc/rc.linkup: ROUTING: entering configure using 'lan'
2020-02-10T12:57:03 dhcp6c[62879]: restarting
2020-02-10T12:57:03 opnsense: /usr/local/etc/rc.linkup: HOTPLUG: Configuring interface lan
2020-02-10T12:57:03 opnsense: /usr/local/etc/rc.linkup: DEVD Ethernet attached event for lan
2020-02-10T12:57:02 kernel: bce3: link state changed to UP
2020-02-10T12:57:02 kernel: bce3: Gigabit link up!
## LAN Comes back up ##
2020-02-10T12:57:01 dhcp6c[62879]: Sending Solicit
2020-02-10T12:57:00 dhcp6c[62879]: Sending Solicit
2020-02-10T12:57:00 kernel: in_scrubprefix: err=51, prefix delete failed
2020-02-10T12:57:00 dhcp6c[62879]: restarting
2020-02-10T12:57:00 opnsense: /usr/local/etc/rc.linkup: DEVD Ethernet detached event for lan
2020-02-10T12:57:00 kernel: bce3: link state changed to DOWN
## Two minutes later I physically DOWN the LAN/WAN interfaces ##
2020-02-10T12:55:37 dhcp6c[62879]: Sending Solicit
## Unable to pass any traffic through any interface ##
2020-02-10T12:54:33 openconnect[30051]: Session terminated by server; exiting.
2020-02-10T12:54:33 kernel: ocvpn0: link state changed to DOWN
2020-02-10T12:54:32 openconnect[30051]: Received server disconnect: b0 'Idle Timeout'
## Server Sends Disconnect ##
2020-02-10T12:53:36 dhcp6c[62879]: Sending Solicit
2020-02-10T12:51:37 dhcp6c[62879]: Sending Solicit
## Traffic is running normally ##

**Additional context**
Firewall is configured with 3 physical interfaces, configured as follows:

• WAN (PPPoE0 on VLAN201) bce2
• LAN (untagged) on bce3
• Servers (untagged) on bce1
• Servers (VLAN 30) on bce1
• Server (VLAN 40) on bce1

bce2 not connected.

**Environment**

• OPNsense 20.1-amd64
• FreeBSD 11.2-RELEASE-p16-HBSD
• OpenSSL 1.1.1d 10 Sep 2019
• Intel(R) Core(TM) i3-2120 CPU @ 3.30GHz (4 cores)
• Dell R210 ii

[errored out]

I am also running into this issue.  However, I am using OpenVPN and the connections on this FW are clients, not Servers. 

Looks like there is are existing threads on this, but last post was from 2020. 
https://forum.opnsense.org/index.php?topic=20045.0 

Additional information, but no responses.
https://forum.opnsense.org/index.php?topic=21533
https://forum.opnsense.org/index.php?topic=15818.0