Hi all,
I just want to share with you some tunables that fixed repetitive crashes when using zenarmor in native mode, there is a warning that say you should be using the emulated mode, but that wasn't enough of an explanation for me, queuese would run out after a few hours and force an interface restart, after a lot of AI fu I was finally given the correct value, these won't work from the web interface as those are applied once the firewall has started which is already late, this works with i350-t4 cards :
cat /boot/loader.conf.local
dev.igb.0.iflib.override_ntxds="4096"
dev.igb.1.iflib.override_ntxds="4096"
dev.igb.2.iflib.override_ntxds="4096"
dev.igb.3.iflib.override_ntxds="4096"
dev.igb.0.iflib.override_nrxds="4096"
dev.igb.1.iflib.override_nrxds="4096"
dev.igb.2.iflib.override_nrxds="4096"
dev.igb.3.iflib.override_nrxds="4096"
dmesg confirms the application of the new values :
[31] tun0: changing name to 'zen0'
[33] igb1: link state changed to UP
[34] igb0: link state changed to UP
[34] igb2: link state changed to UP
[35] igb3: link state changed to UP
[42] 137.783848 [ 805] iflib_netmap_config txr 6 rxr 6 txd 4096 rxd 4096 rbufsz 2048
[42] 137.783906 [ 805] iflib_netmap_config txr 6 rxr 6 txd 4096 rxd 4096 rbufsz 2048
[42] igb1: link state changed to DOWN
[42] igb0: link state changed to DOWN
[43] 138.429571 [ 805] iflib_netmap_config txr 6 rxr 6 txd 4096 rxd 4096 rbufsz 2048
[43] 138.540666 [ 805] iflib_netmap_config txr 6 rxr 6 txd 4096 rxd 4096 rbufsz 2048
[47] igb0: link state changed to UP
[47] igb1: link state changed to UP
as soon as I applied this, my issues were gone.
i would say i found similar over igb, igc, ixl, ax (the opnsense hardware amd 10g driver not the freebsd usb driver)
emulated mode i always got high latency spikes and bad bufferbloat tests. native mode i can get an "A" without any shaping, but i was constantly running into kernel messages about transmit full, etc. this is with a DEC3920 and 1 gbps WAN, so hardware is overkill.
i do
dev.ax.0.iflib.override_nrxds=4096
dev.ax.0.iflib.override_ntxds=4096
and my bufferbloat score moves to "A" and never see any of those transmit full messages in dmesg:
Notice kernel 440.906116 [4335] netmap_transmit ax0 full hwcur 902 hwtail 1016 qlen 909
The reason why the advised deployment is to use emulated Netmap driver is that the Devs on ZA and OPN did advances on the emulated netmap driver.
I run with it for a half year and in my case it outperforms the native one both in performance and latency.
Regards,
S.
i guess my experience has been the exact opposite with emulated mode. they improved emulated mode to not be as awful as it previously was, but i don't recall any statements saying they closed the gap. their own documentation states:
https://www.zenarmor.com/docs/guides/deployment-modes#b-with-emulated-netmap-driver
Be noted that emulated driver is not as performant as the native netmap driver.
i think if you have a sh*t drive like realtek, sure, but the netmap implementation in the intel driver is solid imo.
Yea they should definitely revisit the docs.
But all round its like you said.
The benefit of emulated is that you can run it on HW that does not have a native support of Netmap in the first place.
Regards,
S.