Messages

1

Zenarmor (Sensei) / Re: Sensei on OPNsense - Application based filtering

« on: December 13, 2022, 06:10:31 pm »

my bad, this is a known issue with netmap on lagg interfaces.
https://forum.opnsense.org/index.php?topic=24015.0

To be sure I

• removed lagg, moved LAN to igb0 (netmap issue)
• Moved WAN from re0 to igb1 (realtek driver history)

Everything seems to work as normal again

2

Zenarmor (Sensei) / Re: Sensei on OPNsense - Application based filtering

« on: December 11, 2022, 04:16:40 pm »

Hi,

You can reinstall database by following the below document. Can you share a bug report before DB reinstall to look into the elasticsearch issue?


https://www.sunnyvalley.io/docs/troubleshooting/reporting#how-do-i-reinstall-the-reporting-database

Send the bug-report via Sensei plugin, was happy to see that logs can be included really easily! (Added a link to my comment in the report)
 There are quite a lot of logs in /urs/local/sensei/active ***, do you want them all here too?

To make sure it wasn't a one off crash, I enabled Sensei without enabling the ES service. After ~3 minutes the network had a small crash of ~1 minute, came back up and the system crashed ~2 minutes later (although I can't find anything in the logs).
I shut the system off via the hardware button, waited for a few minutes and booted it up again. Ram usage after boot was ~4gb and reached 6gb when I stopped the Zenarmor engine. ES is still running and ram seems stable ~4gb.

I have removed the database manually, doing the wizard again now. Will update my comment afterwards.


Configuration
WAN: re0, Realtek RTL8111HSD-CG
LAN: lagg0(), 2-port LACP on intel i340-t2

OPNsense community-repo: mimugmail [update1]

ZenArmor
General:
 Mode: Routed with native netmap driver
 Interface: LAN
 DB: ES
 size: Small II (< 51 devices), sensei's doc [1] estimates a throughput of 500 Mbps for this setup with a min. of 4gb.
 
Cloud Threat intel:
 Enabled: yes

Updates & Health:
 Max. Swap Util: 60% *

Reporting & Data:
 Size of the Fast Temporary Memory Disk: 48% **
 Real-time DNS reverse queries for local IP: Disabled
 OPNsense Host aliases for DNS enrichment: Disabled
 Maximum number of days to store reporting data: 7 days


* SWAP is disabled on OPN, does this setting interfere with that? (I assumed the setting is being ignored)
** The default setting. This metric does not include the ES service itself right? (as in, the whole sensei service memory usage). My system uses 1.5gb avg, so ~2gb, add 4gb for fast temp mem disk and I've got only ~2gb left for Sensei?
*** main_, periodical_, seneigui, idpr*_,streamer_, worker_ and update_check.

1. https://www.sunnyvalley.io/docs/introduction/hardware-requirements#cpu--memory

UPDATE 1: Wizard: reporting & database.
During database selection I got the following notification

It looks like you also have mimugmail community repo enabled. Please be advised that this repo is also serving Elasticsearch and Mongodb packages with their dependencies. In this regard sunnyvalley and community repositoriees (spelling error in modal, if a Sensei dev is reading this) are not compatible when enabled at the same time.

If you would like to continue using both repositories, we advise to install Elasticsearch from the community repository and point zenarmor to this database as a "Remote Elasticsearch" database.

My dashboard shows that ES is still running, so I'm going to remove ZenArmor, add mimugmail-ES, install ZenArmor, external source for ES. Will update again.

Also, for my usecase e.g. low user count, relatively low usage, is ES that beneficial compared to Mongo? I'd like to also run OPNsense IDS (suricata) which doesn't really feel feasible right now.

UPDATE 2: ZenArmor ES & community plugins
An existing issue in the plugin repo, https://github.com/mimugmail/opn-repo/issues/116.
I'm already using AdGuardHome & speedtest from the repo, I'm going to offload it to another machine, remove the community repo and try again (bummer that they don't work together though, I was thinking of using some of his plugins)

3

Zenarmor (Sensei) / Re: Sensei on OPNsense - Application based filtering

« on: December 11, 2022, 10:53:05 am »

Thanks for helping out!

I believe I had followed the contents of that guide, but I will try it again this afternoon.
Will look into the logs as well and share them here

4

Zenarmor (Sensei) / Re: Sensei on OPNsense - Application based filtering

« on: December 10, 2022, 12:35:31 pm »

whats the recommended DB type? mongodb? ES? remote ES? i have a home lic with around ~100 users (mostly IoT)

Depends on the hardware used for OPNsense.

When I installed Sensei, it selected ES as default for me, with no possibility to change it manually from the gui. It has been running fine for about 4 days, but since yesterday, after starting Sensei RAM usage linearly grows until the machine is out of ram and crashes (OPN IDS does that too).

I’m running OPNsense on an Optiplex 3070 sff, i5-9500, 8GB ram, 128gb nvme. At most concurrent 2 users, ~15 devices of which ~7 iot. 1000/50 down/up, no external connection/vpn or publicly available content.
My hardware meets OPNsense’s recommended requirements and Sensei’s minimum requirements (constrained by ram), should I just add another 8GB ram stick or could there be some other culprit or known bug I don’t know about?

Is it possible to manually set the database to Mongo? I’ve seen some threads where it’s suggested to change some of the values in the script which selects the db based on hardware, but the posts are 3+ years old and aimed at going from Mongo to ES on hardware which was not recommended