opnsense freezes and needs reboot

[mgrue]

I never went to the commandline with 20.7 to check the ifconfig output. I will wait until opnsense 20.7 is based at least on FreeBSD 12.1-RELEASE-p8 and then re-try (and then also check / play with ifconfig). Thanks for the hints regarding the vmxnet driver.

[rogge+opnsense]

I am also having same issue and very similar configuration (ESXi & VM)...

What about a Periodic interface reset - rather than a system reboot?

a periodic interface reset is scheduled daily on my setup; i'll post results in a week so.

Note: i am using e1000 network 'cards'

[Fright]

thanks! intresting..thought that the transition to e1000 should help
can you please share more info about config: IPS? offloads? plugins?

[rogge+opnsense]

so periodic interface reset did not work. - I am also now cronning a reboot.

DHCP4 -V6 is disabled - all 'static' IP are ARPed
OpenDNS is enabled
UnboundDNS w/ blocklists.
IPS is on using Hyperscan.
All offloads are offloading
only using VMware plugin

[Fright]

since you use IPS have you tried ?
https://forum.opnsense.org/index.php?topic=19175.0

[mgrue]

I have now re-upgraded to 20.7.3 and giving it a try with vSphere/vmxnet drivers.

This is the ifconfig output:

Code: [Select]

vmx0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
        options=800028<VLAN_MTU,JUMBO_MTU>
        ether 00:0c:29:2d:79:14
        inet 192.168.179.1 netmask 0xffffff00 broadcast 192.168.179.255
        inet6 fe80::20c:29ff:fe2d:7914%vmx0 prefixlen 64 scopeid 0x1
        inet6 2003:dd:2f1b:f804:20c:29ff:fe2d:7914 prefixlen 64
        media: Ethernet autoselect
        status: active
        nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>


[mgrue]

After 3 days of uptime the system again stopped forwarding packets.

Obviously it ran out of memory (see screenshot). I restarted all services through SSH, didn't not help.
Rebooted and latencies on the WAN were super high again and the system was extremely sluggish.

I downgraded to 2.1.9 again and all is fine again. There seems to be a problem with 20.7 in vSphere VM and it does not seem related to the vmxnet driver. Any further ideas?

[Supermule]

What version of VM are you running??

After 3 days of uptime the system again stopped forwarding packets.

Obviously it ran out of memory (see screenshot). I restarted all services through SSH, didn't not help.
Rebooted and latencies on the WAN were super high again and the system was extremely sluggish.

I downgraded to 2.1.9 again and all is fine again. There seems to be a problem with 20.7 in vSphere VM and it does not seem related to the vmxnet driver. Any further ideas?

[mgrue]

What version of VM are you running??

VM-Version 14 on ESXi 6.7

[Supermule]

Can you downgrade to version 10??

What version of VM are you running??

VM-Version 14 on ESXi 6.7

[mgrue]

Can you downgrade to version 10??

No, I can't. How can that help? 2.1.9 runs happily with VM version 14.

[Supermule]

Try it.

Backup config and install an OPNsense instance in VM version 10 and report back.

Can you downgrade to version 10??

No, I can't. How can that help? 2.1.9 runs happily with VM version 14.

[GreenMatter]

I won't help much, just to let you compare. I have recently upgraded to 20.7.3 and so far, so good...
Difference is that I use ESXi 7.0, all HW offloading is enabled and OPNsense is VLAN aware; vmx0 is WAN and vmx1 is VLAN parent for LAN side:

vmx1: flags=8a43<UP,BROADCAST,RUNNING,ALLMULTI,SIMPLEX,MULTICAST> metric 0 mtu 1500
   options=e507bb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,TSO4,TSO6,LRO,VLAN_HWFILTER,VLAN_HWTSO,RXCSUM_IPV6,TXCSUM_IPV6>
   ether 00:0c:29:d4:ba:59
   inet6 fe80::20c:29ff:fed4:ba59%vmx1 prefixlen 64 scopeid 0x2
   media: Ethernet autoselect
   status: active
   nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>

Attached are screenshots of vm switch and port group settings...

[mgrue]

Difference is that I use ESXi 7.0, all HW offloading is enabled and OPNsense is VLAN aware;

This made me think. I enabled all offloading capabilities including VLAN filtering and now the system is up and running for nearly 10 days. Thanks for putting me into the right direction.

Soon I will move the opnsense VM to a new ESXi 7.0 U1 box with a more powerful CPU, a 10 Gbit NIC and more RAM. Lets keep fingers crossed that the stability will stay.

[mgrue]

I have ported the whole thing to ESXi 7.0 U1 on a latest generation i3 Processor. The system is now up and running with 20.7.4 since it has been released. I have assigned more RAM to the VM (1.5 GB instead of 1 GB).
All offloading capabilites have enabled with a Broadcom 57810 10 Gig NIC behind the vSwitches. Runs great so far.