opnsense freezes and needs reboot

Started by mgrue, August 18, 2020, 09:14:34 AM

Previous topic - Next topic
I never went to the commandline with 20.7 to check the ifconfig output. I will wait until opnsense 20.7 is based at least on FreeBSD 12.1-RELEASE-p8 and then re-try (and then also check / play with ifconfig). Thanks for the hints regarding the vmxnet driver.

I am also having same issue and very similar configuration (ESXi & VM)...

What about a Periodic interface reset - rather than a system reboot?

a periodic interface reset is scheduled daily on my setup; i'll post results in a week so.

Note: i am using e1000 network 'cards'

September 16, 2020, 10:02:37 PM #17 Last Edit: September 16, 2020, 10:56:39 PM by Fright
thanks! intresting..thought that the transition to e1000 should help
can you please share more info about config: IPS? offloads? plugins?

so periodic interface reset did not work. - I am also now cronning a reboot.

DHCP4 -V6 is disabled - all 'static' IP are ARPed
OpenDNS is enabled
UnboundDNS w/ blocklists.
IPS is on using Hyperscan.
All offloads are offloading
only using VMware plugin


I have now re-upgraded to 20.7.3 and giving it a try with vSphere/vmxnet drivers.

This is the ifconfig output:
vmx0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
        options=800028<VLAN_MTU,JUMBO_MTU>
        ether 00:0c:29:2d:79:14
        inet 192.168.179.1 netmask 0xffffff00 broadcast 192.168.179.255
        inet6 fe80::20c:29ff:fe2d:7914%vmx0 prefixlen 64 scopeid 0x1
        inet6 2003:dd:2f1b:f804:20c:29ff:fe2d:7914 prefixlen 64
        media: Ethernet autoselect
        status: active
        nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>

After 3 days of uptime the system again stopped forwarding packets.

Obviously it ran out of memory (see screenshot). I restarted all services through SSH, didn't not help.
Rebooted and latencies on the WAN were super high again and the system was extremely sluggish.

I downgraded to 2.1.9 again and all is fine again. There seems to be a problem with 20.7 in vSphere VM and it does not seem related to the vmxnet driver. Any further ideas?


What version of VM are you running??

Quote from: mgrue on October 02, 2020, 08:43:40 PM
After 3 days of uptime the system again stopped forwarding packets.

Obviously it ran out of memory (see screenshot). I restarted all services through SSH, didn't not help.
Rebooted and latencies on the WAN were super high again and the system was extremely sluggish.

I downgraded to 2.1.9 again and all is fine again. There seems to be a problem with 20.7 in vSphere VM and it does not seem related to the vmxnet driver. Any further ideas?



Quote from: Supermule on October 02, 2020, 09:02:10 PM
Can you downgrade to version 10??
No, I can't. How can that help? 2.1.9 runs happily with VM version 14.

Try it.

Backup config and install an OPNsense instance in VM version 10 and report back.

Quote from: mgrue on October 02, 2020, 09:12:40 PM
Quote from: Supermule on October 02, 2020, 09:02:10 PM
Can you downgrade to version 10??
No, I can't. How can that help? 2.1.9 runs happily with VM version 14.

I won't help much, just to let you compare. I have recently upgraded to 20.7.3 and so far, so good...
Difference is that I use ESXi 7.0, all HW offloading is enabled and OPNsense is VLAN aware; vmx0 is WAN and vmx1 is VLAN parent for LAN side:

Quote
vmx1: flags=8a43<UP,BROADCAST,RUNNING,ALLMULTI,SIMPLEX,MULTICAST> metric 0 mtu 1500
   options=e507bb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,TSO4,TSO6,LRO,VLAN_HWFILTER,VLAN_HWTSO,RXCSUM_IPV6,TXCSUM_IPV6>
   ether 00:0c:29:d4:ba:59
   inet6 fe80::20c:29ff:fed4:ba59%vmx1 prefixlen 64 scopeid 0x2
   media: Ethernet autoselect
   status: active
   nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
Attached are screenshots of vm switch and port group settings...

OPNsense on:
Intel(R) Xeon(R) E-2278G CPU @ 3.40GHz (4 cores)
8 GB RAM
50 GB HDD
and plenty of vlans ;-)

Quote from: GreenMatter on October 03, 2020, 02:32:55 AM
Difference is that I use ESXi 7.0, all HW offloading is enabled and OPNsense is VLAN aware;

This made me think. I enabled all offloading capabilities including VLAN filtering and now the system is up and running for nearly 10 days. Thanks for putting me into the right direction.

Soon I will move the opnsense VM to a new ESXi 7.0 U1 box with a more powerful CPU, a 10 Gbit NIC and more RAM. Lets keep fingers crossed that the stability will stay.

I have ported the whole thing to ESXi 7.0 U1 on a latest generation i3 Processor. The system is now up and running with 20.7.4 since it has been released. I have assigned more RAM to the VM (1.5 GB instead of 1 GB).
All offloading capabilites have enabled with a Broadcom 57810 10 Gig NIC behind the vSwitches. Runs great so far.