Netmap cap for all or just 10G ix nics?

[gauthig]

One one build I confirmed that the new netmap kernel and 20.7.4 works with ESXI vmx drivers well, but not full speed, stops around 2.5gbs from VM to opnsense VM.  But other VM to VM on same ESXI rans about 20Gbs.
This actually is good for the purpose we need.

On a barebones firewall we have a dual 10G for the lan (intel ix) and netmap (IDS Enabled) seems to bring it down.   

Iperf3 from internal server to opnsense LAN

IDS off  (CPU shows around 5%)
Send
[ ID] Interval           Transfer     Bitrate         Retr
[  5]   0.00-5.00   sec  3.41 GBytes  5.85 Gbits/sec    0           
Receive (-R)
[ ID] Interval           Transfer     Bitrate         Retr
[  5]   0.00-10.14  sec  4.42 GBytes  3.74 Gbits/sec    0           


IDS On - Hyperscan (CPU  40 - 50%)
Send
[ ID] Interval           Transfer     Bitrate         Retr
[  5]   0.00-5.00   sec   742 MBytes  1.25 Gbits/sec    0
Receive (-R)
[ ID] Interval           Transfer     Bitrate         Retr
[  5]   0.00-5.14   sec   455 MBytes   742 Mbits/sec    0

IDS On - Ken Steele (CPU 20-40%)
Send
[ ID] Interval           Transfer     Bitrate         Retr
[  5]   0.00-5.00   sec   272 MBytes   456 Mbits/sec    1           
Receive (-R)
[ ID] Interval           Transfer     Bitrate         Retr
[  5]   0.00-5.14   sec   307 MBytes   502 Mbits/sec    0

Notes

• Server to Server on same 10G network with same 10G cards runs about 9.8gbs so network so no issue on switches or network cards.
• This is being used as an internal server segmentation firewall so I need the full 10G, it is not internet facing.
• Tried the other sense product, got about 7.8gbs on latest version which still users FreeBSD 11 (with SNORT on IPS)
• While opnsens 20.7.4 seems to have issues with ix0 and netmap. still does not seem full speed with ixo and no netmap.
• When enabling IDS, it takes about 75 seconds after service is started before you see performance change.  Don't know why.
• Will try to test with 20.1 to see if just new BSD is issue. 

Can anyone else produce results for 10G?

[mimugmail]

Maybe this one will fix it:
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=248652

No idea when it finds its was into OPNsense

[gauthig]

@mimugmail - Thanks for finding that.  It does seem to be the issue and it's listed in a commit for FreeBSD, so once it makes it there we have to wait for the next OPNSense patch cycle.  Maybe Jan. or Feb. unless OPNsense team adds a temp kernel patch like they did with 20.7.3.

[mimugmail]

I will have a talk to Franco and @mb

[gauthig]

Looks like the Call for Netmap testing thread picked up a new kernel for ix nics so moving my results over there:
https://forum.opnsense.org/index.php?topic=17363.0

By the way, as a preview it helps but brings CPU usage way up.

[klamath]

I am using the ixl drivers with IDS enabled, if I disabled promiscuous mode in IDS i could get full speed again.