Topics

1

17.7 Legacy Series / OPNsense (VM running on KVM) interface stops accepting packets from WAN

« on: January 13, 2018, 06:56:05 am »

I have the stable release of OPNsense running in a VM on a KVM host. I am using the e1000 NIC driver with HW offloading disabled in the OPNsense UI. I have a typical two interface setup and have Suricata running in IDS/IPS mode. The internet connection is 200Mbps/20Mbps cable from the cable company.

* Ever 1-2 days, the WAN interface has problems. tcpdump on the interface shows only outbound packets and not a single inbound.
* I have found that running ifconfig em1 down followed ifconfig em1 up restores normal operation (em1 is the WAN interface).
* I have examined every guest and host log file that I can find and haven't found a single error message which lines up with these outages.
* This internet connection is used heavily during the day for normal home office tasks and the interface has yet to have problems during the day.
* When this outage occurs, we are typically watching 1x streaming show (Hulu, Netflix, etc).
* Load on the VM looks low when the outages happen. Plenty of free ram and a low # of connections.
* After bouncing the interface, the VM/OPNsense does fine the rest of the night and the next days. We typically watch hours of streaming after the outage without any further issues.

Any ideas for the cause or how to troubleshoot further? If I can't solve this issue, I will have to switch firewall solutions as troubleshooting network issues in the middle of the night almost every night isn't much fun.

2

Intrusion Detection and Prevention / Intrusion Detection plus IPS enabled plus vrtio = blocked network traffic

« on: December 30, 2017, 09:16:35 pm »

Note: I originally replied to this thread https://forum.opnsense.org/index.php?topic=5173.0 as it describes the same symptoms as I am facing. However, I now see that the thread is in the 17.1 Legacy subforum and I am using the latest stable OPNsense release so this subforum seemed like a better location.

--------
Guest VM running OPNsense 17.7.11. The host is running Proxmox 5.1. Everything was installed in the last few days so a fairly clean, out of the box configuration.

OPNSense VM is configured as an Internet router (standard auto NAT setup), one interface on the LAN and the other interface on the WAN. Both are bridged to dedicated host interfaces. Network devices are set to vrtio in the VM configuration.

Network device HW offloading is disabled in OPNSense. I also have TX offload disabled at the host level using ethtool.

As soon as IPS is enabled, traffic through the router and from the VM itself to the WAN basically stops. tcpdump on the WAN interface shows a few packets but only a very small % of what should be there. Packets reported by tcpdump are truncated, each missing a different number of bytes.

Behavior is the same even with no IPS/IDS rules enabled. Disabling IPS restores full network functionality. If the network devices are switched to E1000, IPS works correctly and network traffic is forwarded/NAT'd but CPU utilization goes up dramatically.

I am upgrading my Internet connection from 50Mbps to 200Mbps next week so concerned OPNSense isn't going to get the job done, at least not without throwing a lot of HW/energy at the problem.

Has anyone found a solution to this problem which appears to be caused when using IPS with vrtio driver/devices?