OPNsense Forum
Archive => 23.1 Legacy Series => Topic started by: credem on June 01, 2023, 02:23:25 pm
-
Hello everybody,
first post here.
I recently setup an OPNSense box (HUNSN RJ03m, Intel N6005, 32GB DDR4, Intel 2.5GbE I226-V) and I am trying to exploit the hardware to the fullest. I am on the latest version (OPNsense 23.1.9-amd64) and I already installed several plugins:
- AdGuard Home
- wireguard for DNS over VPN
- DynamicDNS for Cloudflare DNS updating
- CrowdSec
- ZenArmor
I also enabled RSS (Receive Side Scaling (https://docs.opnsense.org/troubleshooting/performance.html)) even though I see my network driver (igc) is not explicitly supported.
OPNSense documentation mentions:
Only enable this feature if you’re interested in testing it and seeing if it will increase your throughput under high load – such as when using IDS/IPS.
and since I am definitely using IDS/IPS, I guessed why not.
Now, after installing ZenArmor I started receiving popups about having RSS enabled, and that if I run into any issue I should disable it. I looked a bit deeper into it and I am still not sure what to make of it.
SunnyValley only recommends to disable HW offloading (https://www.sunnyvalley.io/docs/guides/disabling-hardware-offloading) but I don't see anything explicit rather than "your mileage may vary". This thread (https://forum.opnsense.org/index.php?topic=29267.0) says that it has indeed an impact on netmap driver.
On the other hand, in this thread (https://forum.opnsense.org/index.php?topic=34179.msg165373#msg165373) the recommendation for igc is to use the emulated netmap driver for ZenArmor.
I am telling you all this because up until now I used
- ZenArmor with RSS enabled
- ZenArmor with native netmap driver enabled on igc
and I didn't notice any issue whatsoever, while I realize now my setup should cause issues big time.
So my question to you is: how do I make sure my setup isn't borked? Is there any test I can perform to be sure I am not running into issues? Because as of now, I did not have any: firewall works, I get maximum speed in both direction on WAN and local networks work flawlessly.
Should I keep living recklessly, or should I behave and disable RSS and switch to emulated netmap driver for ZenArmor?
I like to test things out and since I just started, I can afford making mistakes and tinker my way around the OS.
At the same time I am not persuaded by the fact that everything works fine™ and would love to hear from people more knowledgeable in this area if what I am doing is profoundly wrong.
Thanks.
-
did you find any difficulties? I ask because I think I have found one, and I can reproduce it but I'm unsure if is to do with Zenarmor. Disabling it seems to make the problem go away.
-
did you find any difficulties? I ask because I think I have found one, and I can reproduce it but I'm unsure if is to do with Zenarmor. Disabling it seems to make the problem go away.
actually yes. Since latest major update with revamped UI (I believe it's version 1.14), ZenArmor stopped working for me. As I stated in the original post, it's been working fine ever since, up until very recently.
I thought it could be because of RSS, but I ended up disabling ZenArmor instead, didn't want to reboot to disable RSS :D
In my case, after I finish the guided procedure in the new ZenArmor interface, it tries to start the engine but I get:
Cannot read any worker configuration from workers.map
Did you have a similar experience when ZenArmor stopped working? And just to confirm, were you able to make it work again by disabling RSS?
P.S.: just fyi, I am now running OPNSense 23.7.3, but ZenArmor stopped working while I was on version 23.1.11. Updating to 23.7 first, and then to 23.7.3, did not change the result: I always get the same error in ZenArmor.
-
Ok well, this deserves another post.
I wanted to reproduce the issue once again since you took interest in my post, and give you a screenshot of the error in ZenArmor.
So I followed the guided procedure, installed a SQLite database (more than enough for my needs) but this time, out of frustration ;D I selected native netmap driver.
(https://i.imgur.com/aHW3ogm.png)
I always left everything default just to be sure it deployed, so i never bothered changing this setting, By re-reading my original post, I realized I was using the native driver, so I guessed why not give it a try.
Lo and behold... it worked :o
(https://i.imgur.com/YNWGB61.png)
Notice how I still get the same popup about RSS I was complaining about in my original post.
So thanks for replying: it started a chain of events that culminated in solving my issue ;D, I hope it solves yours too.
-
Spoke too soon.
While the engine is running, there is a problem with the database.
Reports and Live Sessions pages show the following error:
(https://i.imgur.com/IZcWUPE.png)
More of the same in ZenArmor logs:
[FATAL] UpdateTable, Cannot open /usr/local/datastore/sqlite/conn_all.sqlite: database disk image is malformed
In this state I also can't register my node to the ZenArmor cloud console.
Not sure if you encountered this issue as well, but worth mentioning it since it's not over.
-
Last chapter: I was able to solve the problem above by nuking the reporting database in ZenArmor settings.
Settings --> Data Management --> Reporting Database Settings --> Reset Database and make sure to tick Re-install database.
After resetting and reinstalling the SQLite db, now everything works and I was able to register my node in the cloud management portal.
So, it looks like I am able to run ZenArmor with native Netmap driver and RSS enabled, like I was doing before it broke. I'll keep an eye on it and see if everything works in the future.
-
thanks for this.
I have been running Zenarmor with RSS on igbc interfaces for the last few weeks. RSS was left enabled since I opted in to test it last year I think. Then upgraded from 22.7 to 22.3 a few weeks ago, nothing appeared wrong until recently with my son coming back home, he switched on a games console into a port that is normally not in use and trouble started.
All LAN devices on the LAN port (a different one) which connect either wired or wireless via a managed switch & APs suddenly lose name resolution. No clues on dmesg, AdGuard has no logging except queries, nor Unbound logs.
Even name queries from the firewall timed out, that was the only symptom. Timing out from loopback which was baffling.
This has been going on a few times until recently the pattern broke, same symptoms but no second NIC being used.
The only way to recover was to reboot since there were no clues what service in the DNS resolution chain was having a problem, and both my wife and I are working from home on tight deadlines that require network connectivity.
Until yesterday I could take a bit longer to dig around and I could see during timeouts from loopback that there was a large number of connections open and appearing stalled, all with some name that suggested Zenarmor. They might have been eastpect or similar. As soon as I stopped ZA, naturally they all disappeared and timeouts from the dns service on loopback went away.
Zenarmor has now been left disabled and not starting at boot and although no reboot has been made, the problem has gone away.
This said, ZA does say very clearly in the recent version from 1.14 I think "t looks like you've enabled RSS (Receive Side Scaling) kernel support. Please be noted that RSS support is quite new. If you experience throughput problems, please disable RSS and try again. "
So I thought from the notice that it could cause throughput problems. I take that as reducing bandwidth not a complete failure, hence I left it.
I have submitted a "ticket" from the plugin and see if what they suggest.
Thanks again.
-
Hey, thanks for the feedback.
Unfortunately it seems like your problem is different than mine and I haven't run into a similar issue so far.
I am afraid I can't help with what you are experiencing.
A couple of things I can add:
- You can try to change Netmap driver mode (vanilla, native or emulated)
- I have been pleased so far with SunnyValley support, I believe they can help you out
- Also make sure to check the dedicated thread (https://forum.opnsense.org/index.php?topic=24409.0) about RSS experiments and trials, maybe someone more knowledgeable can help you out.
Hope you can solve your issue.
-
Yes it is what seems a different problem. Thanks for the inputs.