OPNsense Forum
Archive => 17.7 Legacy Series => Topic started by: xupetas on October 17, 2017, 08:50:34 pm
-
Hello All.
I've installed the latest version of opnsense and tryed to run suricata.
My HW is a 3 vCore QEMU/KVM (tryed on qemu 2.7 and 2.9) with 2GB of ram and several VIRTIO NICs.
I've pulled the latest rules, and checked if there was updates to be installed.
Both suricata and opnsense are on the latest version.
When i boot suricata in IPS configuration, on the WAN interface, i loose conectivity.
Here's a ping form the firewall to the next hop to demonstrate what appends:
64 bytes from 192.168.1.254: icmp_seq=0 ttl=64 time=1.135 ms
64 bytes from 192.168.1.254: icmp_seq=1 ttl=64 time=0.991 ms
64 bytes from 192.168.1.254: icmp_seq=2 ttl=64 time=0.595 ms
64 bytes from 192.168.1.254: icmp_seq=3 ttl=64 time=0.749 ms
64 bytes from 192.168.1.254: icmp_seq=4 ttl=64 time=0.828 ms
64 bytes from 192.168.1.254: icmp_seq=5 ttl=64 time=0.803 ms
64 bytes from 192.168.1.254: icmp_seq=6 ttl=64 time=0.766 ms
64 bytes from 192.168.1.254: icmp_seq=7 ttl=64 time=68606.310 ms < Suricata boot
64 bytes from 192.168.1.254: icmp_seq=8 ttl=64 time=68300.215 ms
64 bytes from 192.168.1.254: icmp_seq=10 ttl=64 time=67848.197 ms
64 bytes from 192.168.1.254: icmp_seq=14 ttl=64 time=68108.471 ms
64 bytes from 192.168.1.254: icmp_seq=17 ttl=64 time=67052.875 ms
64 bytes from 192.168.1.254: icmp_seq=18 ttl=64 time=67528.361 ms
64 bytes from 192.168.1.254: icmp_seq=19 ttl=64 time=68002.670 ms
64 bytes from 192.168.1.254: icmp_seq=22 ttl=64 time=67999.273 ms
64 bytes from 192.168.1.254: icmp_seq=23 ttl=64 time=67995.854 ms
There are NO rules loaded for this test, I just started the daemon.
If i stop the IPS mode (active defense) it goes back to normal:
64 bytes from 192.168.1.254: icmp_seq=10 ttl=64 time=38098.266 ms
64 bytes from 192.168.1.254: icmp_seq=15 ttl=64 time=36568.727 ms
64 bytes from 192.168.1.254: icmp_seq=16 ttl=64 time=36987.789 ms
64 bytes from 192.168.1.254: icmp_seq=20 ttl=64 time=37390.267 ms
64 bytes from 192.168.1.254: icmp_seq=21 ttl=64 time=36703.024 ms
64 bytes from 192.168.1.254: icmp_seq=24 ttl=64 time=36385.351 ms
64 bytes from 192.168.1.254: icmp_seq=26 ttl=64 time=51639.619 ms
64 bytes from 192.168.1.254: icmp_seq=27 ttl=64 time=52612.838 ms
64 bytes from 192.168.1.254: icmp_seq=29 ttl=64 time=51815.528 ms
64 bytes from 192.168.1.254: icmp_seq=30 ttl=64 time=51632.411 ms
64 bytes from 192.168.1.254: icmp_seq=32 ttl=64 time=50433.146 ms
64 bytes from 192.168.1.254: icmp_seq=85 ttl=64 time=0.504 ms
64 bytes from 192.168.1.254: icmp_seq=86 ttl=64 time=0.473 ms
64 bytes from 192.168.1.254: icmp_seq=87 ttl=64 time=0.462 ms
64 bytes from 192.168.1.254: icmp_seq=88 ttl=64 time=0.404 ms
64 bytes from 192.168.1.254: icmp_seq=89 ttl=64 time=0.502 ms
64 bytes from 192.168.1.254: icmp_seq=90 ttl=64 time=0.444 ms
64 bytes from 192.168.1.254: icmp_seq=91 ttl=64 time=0.474 ms
Am i missing something or is this a bug?
I need the IPS active to do active defense on my system?
Or can it be run with active defence on legacy configuration thru pcap thus being only and IDS and not a IPS?
Thanks,
Nuno
-
Hi,
I have the exact same 'issue'.
And do you also see a drop in speed? I have a 400 Mb/s subscription, but with Suricata enabled (in IPS mode with 9 rulesets) I get around 120 Mb/s. And when I disable the Intrusion Detection service it immediately goes back to around 400 Mb/s
-
Hi Nuno,
See if e1000 driver emulation helps... IPS mode is built around Netmap, which like its cousin DPDK is strongest on actual hardware. VMs can be tricky or less performant.
@scrensen, that does not seem to be the same issue. I'm sure you have a number of rules enabled and packets are not dropped. Not knowing anything about your hardware and software setup, I'd say it looks to be within the realm of general possibility.
Cheers,
Franco
-
I have changed form the virtio vnic to the e1000 vnic.
It works perfectly now as expected.
Thanks for your help and feedback!
-
Hello.
I have exactly the same issue with my virtual OPNsense, even if I switch to E1000. All of the activated Interfaces (inside IDS configuration) won't work if I enable IPS. :(
This behaviour I have with VIRTIO & E1000. Just with VMXNET3 it seems to works. But here I loose ~70% of the physical Bandwidth after activating IDS/IPS. (even if I have increased CPU Cores & Memory from this Virtual Machine)
It would be great if you have any Idea how to realize OPNsense inside a virtual Machine with a good performance and activated IPS.
Thanks,
Fabio
-
Same thing with us, I'm afraid. No problem without ips. With ips massive problems. No matter which driver. (tested virtio and e1000). I'm at a loss, too. It works in front of all with pfsense. Been wanting to replace our big pfsense for a long time. Doesn't make any real sense without ips.
Regards
Christian
Gesendet von meinem Redmi Note 4 mit Tapatalk
-
Er, does this pfSense do native IPS (netmap) mode on the same virtualisation host? If yes, which version?
-
Hello all,
I am running my opnsense firewall cluster on two differente versions of KVM/QEMU:
qemu-2.7.0-360
qemu-2.9.0-385
Both of them work pefectly now with IPS and the em0 on the interface that is going to be using the IPS feature (on my case the WAN)
On wich virtualization engine you are having issues?
Thanks!
Nuno
-
For me the same
its kills the connections from 1Gbps/s to 200Mbps.
I am on a hardware
hardware offload is disabled but its still killing my connection.
(https://docs.opnsense.org/_images/disable_offloading.png)
-
Hello Nuno.
Thank you for this information.
I am currently running pve-qemu-kvm: 2.9.0-pve4 @ my PROXMOX PVE Host.
Fabio
-
For me the same
its kills the connections from 1Gbps/s to 200Mbps.
I am on a hardware
hardware offload is disabled but its still killing my connection.
(https://docs.opnsense.org/_images/disable_offloading.png)
Hello Julien,
Is the difficulty being shown at any speed? Or is only felt when you reach 200mbps?
Thanks!
-
Hello Julien,
Is the difficulty being shown at any speed? Or is only felt when you reach 200mbps?
Thanks!
In my environment I tested it via iperf to a System in another Subnet or over WAN:
Virtio and IPS disabled: ~900Mbit/s
E1000 and IPS disabled: ~200-250Mbit/s
VMXNET3 and IPS disabled: ~500Mbit/s
VMXNET3 and IPS enabled: ~300-400Mbit/s
Grretings,
Fabio
-
Hello Julien,
Is the difficulty being shown at any speed? Or is only felt when you reach 200mbps?
Thanks!
In my environment I tested it via iperf to a System in another Subnet or over WAN:
Virtio and IPS disabled: ~900Mbit/s
E1000 and IPS disabled: ~200-250Mbit/s
VMXNET3 and IPS disabled: ~500Mbit/s
VMXNET3 and IPS enabled: ~300-400Mbit/s
Grretings,
Fabio
Hello again.
My problem was not the same as yours. My connections simply dropped dead. No even ping would pass.
It appears that is issue on your case is the vCPU of the opnsense appliance is not strong enough to handle all the traffic and IPS at the same time.
Can you add other vCPU's to your vm, and pin them with exclusivity to a physical CPU on the host and re-check?
Thanks
-
Hello Julien,
Is the difficulty being shown at any speed? Or is only felt when you reach 200mbps?
Thanks!
In my environment I tested it via iperf to a System in another Subnet or over WAN:
Virtio and IPS disabled: ~900Mbit/s
E1000 and IPS disabled: ~200-250Mbit/s
VMXNET3 and IPS disabled: ~500Mbit/s
VMXNET3 and IPS enabled: ~300-400Mbit/s
Grretings,
Fabio
Hello again.
My problem was not the same as yours. My connections simply dropped dead. No even ping would pass.
It appears that is issue on your case is the vCPU of the opnsense appliance is not strong enough to handle all the traffic and IPS at the same time.
Can you add other vCPU's to your vm, and pin them with exclusivity to a physical CPU on the host and re-check?
Thanks
Yes, I have the same issue with E1000 and VirtIO, if IPS is enable -> nothing works!
And of course - I still played with the amount of vCores and with the emulated CPU Type. But it is always the same with E1000 and VirtIO. :(
I am running my Installation currently with VMXNET3, because I don't want to give up IPS...
-
Hello Julien,
Is the difficulty being shown at any speed? Or is only felt when you reach 200mbps?
Thanks!
In my environment I tested it via iperf to a System in another Subnet or over WAN:
Virtio and IPS disabled: ~900Mbit/s
E1000 and IPS disabled: ~200-250Mbit/s
VMXNET3 and IPS disabled: ~500Mbit/s
VMXNET3 and IPS enabled: ~300-400Mbit/s
Grretings,
Fabio
Hello again.
My problem was not the same as yours. My connections simply dropped dead. No even ping would pass.
It appears that is issue on your case is the vCPU of the opnsense appliance is not strong enough to handle all the traffic and IPS at the same time.
Can you add other vCPU's to your vm, and pin them with exclusivity to a physical CPU on the host and re-check?
Thanks
Yes, I have the same issue with E1000 and VirtIO, if IPS is enable -> nothing works!
And of course - I still played with the amount of vCores and with the emulated CPU Type. But it is always the same with E1000 and VirtIO. :(
I am running my Installation currently with VMXNET3, because I don't want to give up IPS...
It is normal a drop in traffic when using IPS. It takes the CPU time to process and validade the data.
It's the same on every IDS/IPS system.
On this case, the usage that suricata does of the netmap driver capability makes it more noticeable.
-
Hello Julien,
Is the difficulty being shown at any speed? Or is only felt when you reach 200mbps?
Thanks!
In my environment I tested it via iperf to a System in another Subnet or over WAN:
Virtio and IPS disabled: ~900Mbit/s
E1000 and IPS disabled: ~200-250Mbit/s
VMXNET3 and IPS disabled: ~500Mbit/s
VMXNET3 and IPS enabled: ~300-400Mbit/s
Grretings,
Fabio
Thank you Fabio,
the connections works only its dropped significly down.
i tried everything but nothing helped.
-
Thank you Fabio,
the connections works only its dropped significly down.
i tried everything but nothing helped.
Hello Julien.
What for an Virtualization Host and qemu-Version you are using?
-
Thank you Fabio,
the connections works only its dropped significly down.
i tried everything but nothing helped.
Hello Julien.
What for an Virtualization Host and qemu-Version you are using?
Hi Fabio,
I am on a hardware with
i5 CPU / 8GB Memory / 64 SSD GB HDD/ Intel 82574L Gigabit Ethernet
-
Hi Fabio,
I am on a hardware with
i5 CPU / 8GB Memory / 64 SSD GB HDD/ Intel 82574L Gigabit Ethernet
So, if you are running your OPNsense on Hardware directly -> check out your current Pattern matcher (under Services/IntrusionDetection). For better Performance you could try "Hyperscan" instead of "Aho-Corasick".
Fabio
-
Last time I tried to enable IPS on a VM running in Proxmox (KVM) it would just stop passing traffic and usually need a reboot to get going again, this was with the virtIO drivers.
it was an N3150 so gutless but it wasn't a CPU usage problem it was the VirtIO drivers really don't seem to play nice with IDS.
It's the reason I don't have the IDS turned on in any of my opnsense boxes because with most of them being virtual I can't risk it.
-
Hi Fabio,
I am on a hardware with
i5 CPU / 8GB Memory / 64 SSD GB HDD/ Intel 82574L Gigabit Ethernet
So, if you are running your OPNsense on Hardware directly -> check out your current Pattern matcher (under Services/IntrusionDetection). For better Performance you could try "Hyperscan" instead of "Aho-Corasick".
Fabio
I have tried both Hyperscan and aho now is running on Default.
both are providing a poor performance.
with hyperscan I reach 400 Mbps and with Aho-Corasich and Default I reach 340 Mbps
-
Does anybody has a idea about why the speed is 50% down when Suricata on is ?
-
It depends on your setup. If you use local services like proxies on your OPNsense and all of this traffic hits the proxy, the proxy penalty usage is 50% because you not only end up reading incoming packets, but also rewiriting or recreating packets which caused buffers to be copied for sending, and that hurts your overall performance.
Especially with a HTTP speed test and web proxy enabled you‘re testing your maximum speed as configured, but maybe not as expected. ;)
Cheers,
Franco
-
It depends on your setup. If you use local services like proxies on your OPNsense and all of this traffic hits the proxy, the proxy penalty usage is 50% because you not only end up reading incoming packets, but also rewiriting or recreating packets which caused buffers to be copied for sending, and that hurts your overall performance.
Especially with a HTTP speed test and web proxy enabled you‘re testing your maximum speed as configured, but maybe not as expected. ;)
Cheers,
Franco
Well Explained Franco, I thought too it has something to do with the proxy but we have no proxy configured in this configuration.
the test is going over http http://beta.speedtest.net/ and it shows 320Mbps/s
is was actually wondering if the speedtest result are our actuall speed or not.
when we test using https https://fast.com/en/gb/ its shows 520 Mbps/s