OPNsense Forum

Hello,

I've been running Zenarmor on my DEC3840 for a while and just recently I've upgraded to a 10G/10G p2p INET connection. Zenarmor is monitoring my AX1 and I wonder whether it supports Native Netmap as I'm seeing a growing number of errors (OUT) on all VLANs as reported onto the INTERFACE STATISTICS widget.

I hadn't notice any errors when Zenarmor was monitoring igbx ports.

Any suggestion?

Hi,

Please visit the following link for the HW requirements of Zenarmor.
https://www.zenarmor.com/docs/introduction/hardware-requirements

Zenarmor works sing-core with the current version and can not handle 10 Gbps traffic. How many users do you have and how many is the throughput?

Hello,

I checked that link before posting and it's not helping in my case as I've got just a bunch of users per se (less than 10) an around 150 clients overall.

My FW throughput is 17G supported by an EPYC CPU with 32G ECC RAM

That being said, my question is around whether native netmap is supported on ax1 port (SFP+ module) as I'm setting errors building up in the interface statistic widget of OPNsense.

Any help?

Let me get this straight: Zenarmor is supposed to protect the LAN, not the WAN.

Quote from: almodovaris on June 25, 2024, 03:33:53 PM
Let me get this straight: Zenarmor is supposed to protect the LAN, not the WAN.

Correct, even thou there is a possibility to RUN it on WAN. ZenArmor as product is focused to protect Endpoint on the LAN. Its whole ecosystem targets, scopes and protects Endpoint on the LAN.

Suricata is the recommend way to be used on WAN.

Regards,
S.

Quote from: almodovaris on June 25, 2024, 03:33:53 PM
Let me get this straight: Zenarmor is supposed to protect the LAN, not the WAN.

Let me get this straight: when have I ever mentioned that I was protecting the WAN?????

That 10G interface is the LAN.

UPDATE: I'm getting errors building up on all VLANs running on that IF as well as the physical IF itself

Quote from: NW4FUN on June 25, 2024, 09:17:25 PM

Quote from: almodovaris on June 25, 2024, 03:33:53 PM
Let me get this straight: Zenarmor is supposed to protect the LAN, not the WAN.

Let me get this straight: when have I ever mentioned that I was protecting the WAN?????

That 10G interface is the LAN.

UPDATE: I'm getting errors building up on all VLANs running on that IF as well as the physical IF itself

Can you specify which counter?

Do you see on
VLANs increase on Output Errors
Physical ports Send Queue Max Length

Regards,
S.

Quote from: Seimus on June 25, 2024, 09:32:59 PM

Quote from: NW4FUN on June 25, 2024, 09:17:25 PM

Quote from: almodovaris on June 25, 2024, 03:33:53 PM
Let me get this straight: Zenarmor is supposed to protect the LAN, not the WAN.

Let me get this straight: when have I ever mentioned that I was protecting the WAN?????

That 10G interface is the LAN.

UPDATE: I'm getting errors building up on all VLANs running on that IF as well as the physical IF itself

Can you specify which counter?

Do you see on
VLANs increase on Output Errors
Physical ports Send Queue Max Length

Regards,
S.

Hi Seimus,

Thanks for your support. Please find attached a screenshot of what I'm seeing.

EDIT: ROOT is the physical IF with a /24 management ip, everything else are VLANs running onto that IF

Yea thats too much,

Those statistics I mentioned, you can find them in Interface > Overview > (your physical Interface or LAGG if you have) ROOT

Check as well there if you can.

Regards,
S.

Yeah sure!

Please find attached screenshots for both ROOT (physical IF) and LAN (VLAN living in ROOT)

Thanks for the pics, you as well showed there Send Queue Drops, on which I forgot to ask.

I have seen with ZenArmor >

Usually when I was trying to push traffic above 1G from a LAN inspected interface (by ZenArmor) I have seen that the Send Queue Drops = Output Errors.

The Send Queue Drops I think were related to the fact that too much traffic was going thru but the system was not able to processes it which caused a drop and generated an Output Errors on a VLAN. I was able to mitigate this by turning on RSS.


If you count the Output errors on all your VLANs (dont without the physical interface) does it match the Send Queue Drops on your physical interface?

It would be good if the ZenAmor support team or Devs commented on this.

In regards of your output errors on the physical interface >
ROOT interface, does it have IP address, I mean do you by any chance mix TAGGed VLANs on a UNTAGGed interface?

Regards,
S.

Thanks for taking the time to look into this...

Going in order:

1) Yes, Send Queue Drops = SUM(VLANs Output Errors)
2) When I turn RSS on, the actual errors number goes through the roof
3) Yes, the physical interface has its own IP where switches and APs are living. Is that a mistake?

Alright so my input

1. So you are seeing exactly what I had seen. Same behavior, I believe this is due to the fact that ZenArmor bottlenecks the backplane. Even if in theory or practice you have 10G LAN and without ZenArmor you are able to get that throughput. ZenArmor is using only single core, thus you will see massive bottleneck. Currently I didn't even see a single core CPU capable enough to run 10G with ZenArmor.

2. I forgot to mention you need to check in settings in ZenArmor the "Do not pin to single core" option this with RSS uplifted a bit the performance and I was able to go above 1G, however as ZenArmor is a single CPU product (currently) you will still see an impact on the throughput.

3. I thought so, this maybe explains the reason why there are so many extra Out errors specifically just for this Interface. And if it was a mistake, well I would say yes. You shouldn't mix unTAGGed and TAGGed VLANs like this. Lot of times it causes odd behavior. Additionally in networking if we do VLANs we bind them to a physical port or LAGG but that port or LAGG (parent interface) we do not configure L3 on it.

Regards,
S.

OK, so...

1) for troubleshooting purposes, I've now uninstalled Zenarmor...very little errors on VLANs, but still there

2) I've done done that and did not help unfortunately

3) I've moved that IP from physical IF to a dedicated VLAN

I've also factory reset my Tunables and reconfigured them...still no luck

What would you suggest I shall do for better troubleshoot?

Quote1) for troubleshooting purposes, I've now uninstalled Zenarmor...very little errors on VLANs, but still there

Funny thing is there always will be small amount of errors if you use VLANs on BSD. Usually you will see few come up during boot. Do those errors increase periodically? If not you dont need to bother.

Quote2) I've done done that and did not help unfortunately

Yea sadly that's what I was pointing out, ZenArmor single core, so you will see bottleneck, you will not be able to get 10G.

Quote
3) I've moved that IP from physical IF to a dedicated VLAN

Good, that's how it should be! Did it help for those Output errors on physical Interface?

QuoteWhat would you suggest I shall do for better troubleshoot?

Depends,

If you mean in regards of ZenArmor and capability to get 10G throughput, there is no option now. We need to wait for them to bring the multicore support. I did open a thread on the forum calling them out to give us an ETA and rumbled a bit...

If you mean in regards of errors with ZenArmor, its as I described due to these reasons.

Regards,
S.

Dear Fellow Zenarmor Users,

We had to change multi-core support priority at the beginning of this year for the critical SSE and TLS inspection features. The team is started to work for multicore support and planning to ship at the en of the year. Many thanks to all for your patience and cooperation.

I've now uninstalled Zenarmor for good and going to cancel my subscription at the end of its term (wish I had a refund TBH!!)

Now no more errors on IFs (either physical or VLANs) and performances are gone back up to more than decent.

HOWEVER, what puzzles me is that Speedtest returns just 5G DL/UL when tested at FW level via CLI and 1G clients are capped at 740M DL while UL is fine at 940M.

I may open a new topic somewhere else...but, what do you think might be the cause of this?

I'm running on a Deciso DEC3840 with 64G ECC RAM

 Hi,

Sorry to hear that and hope to see you again when Zenarmor supports multicore.

Do you use any other inspection tool like Suricata or etc?

I get similar speeds on mine with Zenarmor in place (900 up and 700 down), yes it is odd as I've previously clocked more than 900 each way on my connection at work. I'm just leaving it there and not worrying, it's enough for my department's needs right now. I do have IDS/IPS and Crowdsec running along with Zenarmor and all gigabit connections on an Intel i350 4 port card. CPU never really runs above 50% and normally less than 10% when testing. Tested with both Speedtest.net and Fast.com.

I'm setting up a fresh Business level install, maybe I will test before I start adding the protections and see what happens. Might be fun to log each resource and see the impact of each. It's also a different processor (Intel vs AMD for testing device).