VLAN xen HVM crashes

[tokade]

Hi all,

after several days of nightly crashes and some commands / functions, which crashes OPNsense immediately, I testes all day long what could be the reason.

Here is my configuration:

• Ubuntu 18.04.3 LTS (Bionic Beaver) with XEN 4.9.2
  xn0: bridge for WAN (Intel NIC)
  xn1: bridge for LAN (Intel NIC)
  xn2: bridge for DMZ (w/o NIC)
  xn3: bridge for WLAN (Intel NIC)
• OPNsense 19.7.8-amd64 as HVM domU
  3 GB RAM
  40 GB disk (with 4GB swap)
  
• FreeBSD 11.2-RELEASE-p16-HBSD
  
• DHCPD, NTPD, RADVD, UNBOUND
  

Starting tests from LiveCD and then a fresh installed domU. I think I could narrow down the reason for my crashes.

The crashes started, when I use VLANs on the WLAN interface xn3 in the new OPNsense domU again - without them, I couldn't force a crash. The VLANs have been configured in OPNSense with parent interface MTU 1504 and for the WLAN bridge set in dom0 to 1504 either. The VLANs work well and no TX drops in dom0 or domU.

I'm pretty sure now, that he VLANs are the reason for the nightly crashes too, since I introduced them some days before the update to 19.7.8. Here is what i tested and the results for the commands / functions that crashes the domU:

domU from LiveCD mode:
OPNsense 19.7-amd64
FreeBSD 11.2-RELEASE-p11-HBSD
OpenSSL 1.0.2s 28 May 2019

without IPv6 on WAN
no configuration
sysctl -a via CLI: OK
Audit health via GUI: OK
edit pppoe via GUI: OK
---
domU from fresh install:
OPNsense 19.7-amd64
FreeBSD 11.2-RELEASE-p11-HBSD
OpenSSL 1.0.2s 28 May 2019

without IPv6 on WAN
minmal configuration (WAN & LAN)
sysctl -a via CLI: OK
Audit health via GUI: OK
edit pppoe via GUI: OK
----
install os-xen 1.2 and reboot
sysctl -a via CLI: OK
Audit health via GUI: OK
edit pppoe via GUI: OK
----
update --> 19.7.8
OPNsense 19.7.8-amd64
FreeBSD 11.2-RELEASE-p16-HBSD
OpenSSL 1.0.2t 10 Sep 2019

sysctl -a via CLI: OK
Audit health via GUI: OK
edit pppoe via GUI: OK
----
activate DHCPv6 on WAN
sysctl -a via CLI: OK
Audit health via GUI: OK
edit pppoe via GUI: OK
----
activate IPv6 on LAN (track WAN)
sysctl -a via CLI: OK
Audit health via GUI: OK
edit pppoe via GUI: OK
----
restore DHCP, DHCPv6, system tunables, unbound conf from backup and reboot
sysctl -a via CLI: OK
Audit health via GUI: OK
edit pppoe via GUI: OK
---
restore interfaces (with VLANs) from backup and reboot
sysctl -a via CLI: crashes
Audit health via GUI: OK
edit pppoe via GUI: crashes

Hopefully somebody else has a working installation of OPNsense as HVM domU with VLANs. Since I haven't found anything similar here in the forum or with google, any advice what could be wrong with my configuration is appreciated.

A crash report (sysctl -a) was sent after reboot with the last configuration from my test domU. If necessary, I can do more tests or try to debug /  to trace.

Kind regards
Torsten

[tokade]

Today I did test a fresh installation, no update and imported my configuration with the VLANs. The domU crashes immediately with sysctl -a.

Removing the bridge for WLAN in the domU configuration, which is used for the VLANs no crash with sysctl -a.

The bridge uses one of the 4-ports of a Intel i350 card. So I'm wondering, if the combination of xen, i350 and freebsd, vlan is the reason.

Anybody has a similar configuration?

Kinde regards
Torsten