netmap performance

[gauthig]

Revising several threads from the 20.7 forum. Please refer to for details sample of iperf runs:  https://forum.opnsense.org/index.php?topic=17363.msg93234#msg93234

So with 21.1.1 the performance of ix0 and vnet drivers dropped even more without netmap.  But with netmap (turned on with IDS) is unbearable .  I have noticed something really strange when I toggle IDS of then on again.   I drop from just over 3.5gbs to 2.9gbs after I wait 30 seconds for the IDS to enable. But about 5 minutes later is drops down to 600mbs.  I can't even get 1gbs through. The em0 interface goes from 980mbs to about 940mbs which is much better than I expected.  I can leave the machine idle for 10 minutes and still same results with no other traffic flowing.

So it is still an issue with the ix drivers in hardenbsd 12, so I though.  On a whim, I installed the new pfsense 2.5 which has moved to HardenBSD 12 just like OPNsense.  No netmap on ix0 is about 6.7gbs, with suricata on IPS mode (netmap enabled) I get 3.7gbs.  Did same test and let data stream for 10 minutes and saw no change in that number.

@Franco - Is there a test kernel that has netmap or ix driver updates I can test with for 21?  Also, based on that slowdown after 5 minutes and differences between the two bsd distros, do you think there is anything in OPNSense tunable we can look at.

Bottom line, I need get the 10G nics to work at least at 4gbs for our workload. 

[gauthig]

Attached is a screenshot showing the strange behavior of slowing down about 5 minutes later. What I did was turn of IPS,  then started IPS.  Waited 3 minutes for it to be fully up and settled down.  Then started iperf.  You will notice it was a consistent 3gbs but at 59 seconds something stopped traffic.  About 5 seconds later we started getting the 600mbs speed.

This network is a test network and idle so no other loads.   This test is easily reproduceable.