tweaks that fixed my zenarmor native mode

[abenaou]

Hi all,

I just want to share with you some tunables that fixed repetitive crashes when using zenarmor in native mode, there is a warning that say you should be using the emulated mode, but that wasn't enough of an explanation for me, queuese would run out after a few hours and force an interface restart, after a lot of AI fu I was finally given the correct value, these won't work from the web interface as those are applied once the firewall has started which is already late, this works with i350-t4 cards :

Code Select Expand

cat /boot/loader.conf.local
dev.igb.0.iflib.override_ntxds="4096"
dev.igb.1.iflib.override_ntxds="4096"
dev.igb.2.iflib.override_ntxds="4096"
dev.igb.3.iflib.override_ntxds="4096"
dev.igb.0.iflib.override_nrxds="4096"
dev.igb.1.iflib.override_nrxds="4096"
dev.igb.2.iflib.override_nrxds="4096"
dev.igb.3.iflib.override_nrxds="4096"
dmesg confirms the application of the new values :

Code Select Expand

[31] tun0: changing name to 'zen0'
[33] igb1: link state changed to UP
[34] igb0: link state changed to UP
[34] igb2: link state changed to UP
[35] igb3: link state changed to UP
[42] 137.783848 [ 805] iflib_netmap_config       txr 6 rxr 6 txd 4096 rxd 4096 rbufsz 2048
[42] 137.783906 [ 805] iflib_netmap_config       txr 6 rxr 6 txd 4096 rxd 4096 rbufsz 2048
[42] igb1: link state changed to DOWN
[42] igb0: link state changed to DOWN
[43] 138.429571 [ 805] iflib_netmap_config       txr 6 rxr 6 txd 4096 rxd 4096 rbufsz 2048
[43] 138.540666 [ 805] iflib_netmap_config       txr 6 rxr 6 txd 4096 rxd 4096 rbufsz 2048
[47] igb0: link state changed to UP
[47] igb1: link state changed to UP
as soon as I applied this, my issues were gone.

[dirtyfreebooter]

i would say i found similar over igb, igc, ixl, ax (the opnsense hardware amd 10g driver not the freebsd usb driver)

emulated mode i always got high latency spikes and bad bufferbloat tests. native mode i can get an "A" without any shaping, but i was constantly running into kernel messages about transmit full, etc. this is with a DEC3920 and 1 gbps WAN, so hardware is overkill.

i do

Code Select Expand

dev.ax.0.iflib.override_nrxds=4096
dev.ax.0.iflib.override_ntxds=4096
and my bufferbloat score moves to "A" and never see any of those transmit full messages in dmesg:

Code Select Expand

Notice kernel 440.906116 [4335] netmap_transmit ax0 full hwcur 902 hwtail 1016 qlen 909

[Seimus]

The reason why the advised deployment is to use emulated Netmap driver is that the Devs on ZA and OPN did advances on the emulated netmap driver.

I run with it for a half year and in my case it outperforms the native one both in performance and latency.

Regards,
S.

[dirtyfreebooter]

i guess my experience has been the exact opposite with emulated mode. they improved emulated mode to not be as awful as it previously was, but i don't recall any statements saying they closed the gap. their own documentation states:

https://www.zenarmor.com/docs/guides/deployment-modes#b-with-emulated-netmap-driver

Code Select Expand

Be noted that emulated driver is not as performant as the native netmap driver.
i think if you have a sh*t drive like realtek, sure, but the netmap implementation in the intel driver is solid imo.

[Seimus]

Yea they should definitely revisit the docs.

But all round its like you said.
The benefit of emulated is that you can run it on HW that does not have a native support of Netmap in the first place.

Regards,
S.