Sensei on OPNsense - Application based filtering

[ruffy91]

For comparison I get the following throughput with/without sensei on a pcengines APU3A4:
The interface is just the LAN interface which is a igb NIC without VLAN or LAGG.

Without Sensei 250/50 Mbps
With Sensei 140/40 Mbps

I enabled some security features of sensei and I blocked the malware Web category.

I do not use any other features which do have an impact on throughput like IDS or traffic shaping.

[Antaris]

@ruffy91, it's good that enterprise addon even works on APU3A4s CPU(and on top of that - it's free). If you want fluent Sensei, remember few things: full blown Xeon or desktop i5-7 CPU, 8 ram, SSD. For energy efficient platforms will always be heavy performance loss.

[mb]

@thg0432, yes, currently working on it. We'll provide more info on the timing and details early next month.

@manjeet, glad that your problem with the e-mail report is resolved.  it looks like re-configuring the e-mail server settings proved to be a workaround.

However, for the root cause, if anyone out there who has upgraded from 0.7 and experiencing the e-mail reporting problem, we'd like to dig together.

Regarding deployment size, it looks like that sometimes physical memory size is reported less than exact 8GB (e.g. 7.8GB). So we've adjusted the minimum threshold a bit to accommodate that case.

We'll ship 0.8 release tomorrow morning PST. Hopefully it will resolve your situation.

[marcri]

Hi,
some questions about sensei:
- is it possible to use  an existing elasticsearch instance on a dedicated server?
- if it's possible, can I use one elastic-server for two opnsense instances (failover-setup)?
- where can i get information about using sensei on a corporate network? Prices?
Best
Marc

[aimdev]

Issues I encountered after installing Sensei included web interface locking up, and unable to access opnsesne via ssh. I could still interact with the console. After this occurred i had to uninstall the plugin.
Hopefully the information that the plugin sent back to sunnyvalley will provide insight into my web/ssh issues.

I have the same issue, no access to ssh (an operational requirement) however by enabling bypass mode I can access ssh.

I am running the latest beta version, downloaded today.

Fine Tuning the Remote access option (tediously disable all except the Secure Shell option) gets access back.

[mb]

Fine Tuning the Remote access option (tediously disable all except the Secure Shell option) gets access back.

@aimdev, many thanks for the feedback. I guess the confusing thing is we also have a "ssh" application under "General TCPIP" category. We're fixing this with the upcoming 1.0.

@marcri,

For the main database, you cannot use an external database at the moment. Though premium subscription is offering an option to stream reporting data to an "additional" elastic search database via either syslog or native elasticsearch REST API. 

From time to time we get this request. I guess we should start planning on having the database on an external system. When we do that, it should be trivial to have one elastic instance (either clustered or not) serving many Sensei deployments.

Imagine you're an MSP serving multiple clients or you are a corporate having multiple OPNsense deployments. With such a setup, you should be able to have an aggregate big picture view of whole assets in a centralized system. This way, you could also benefit from Kibana and other 3rd party reporting tools.

Today we're releasing 0.8. Next month, we'll ship 1.0, integrated with OPNsense; and with the details of Premium subscription. Stay tuned :)

[mb]

Dear Sensei users,

After six months of ongoing effort & field testing, it's our pleasure to announce that Sensei 0.8 is finally released.

For some of you who were using 0.7, this version brings quite a loaded set of features:
https://www.sunnyvalley.io/post/sensei-0-8-is-released

We will be releasing Sensei 1.0 next month, in July 2019, which will also cease the BETA program and the software will be publicly available for all users.

[Archanfel80]

Wow! This is great! One of the bests and most wanted missing feautures added to our belowed opnsense firewall. Sensei is one of a kind software for sure! Keep up the good work! :)

[mb]

@Archanfel80,

Many thanks for your feedback. With its open, flexible, extendable architecture; and its great community of users, we love working with OPNsense.

We will do our best to keep adding more value.

[manjeet]

HI MB, everything works fine as mentioned after the update.

Now i have 1 issue and 1 feature request (If its not already there)

Issue: I am not able to update sensei package from command line when using the autoupdate of opnsense i.e. option 12. Same thing happened when i upgraded from 0.7 and now same for yesterday's update. I can only update sensei package from sensei dashboard in web gui.

Feature: Is there any way to for an single or multiple websites / app / category to only put in alert mode. For example if i want my network users to allow access to certain websites but also want to know who access the website or protocol and when AND for specific blocked contents i.e. when someone tried to access it and rather then looking for access logs or block logs just simply have a different tab for alerts only to check easy and fast. I know we can filter it on reports but it will be easy to have an alert tab for both allowed and blocked for that specific alert mode. AND Can we also send alerts via email ?

[Archanfel80]

Just a quick report about an issue what i see.
If you installed sensei from the cli first while in the beta and updated since then for some reason the search data not deleted and consumed the disc space after the final 0.8 upgrade. I cant delete the date from the webui it just says simply 'error'.
I cant figured out why but removed the sensei completely, deleted the '/usr/local/sensei' folder and reinstall sensei from the plugins. Now everything works and the disc usage reduced dramatically. So if you're like me, so installed sensei while in the beta probably the best to backup the config remove sensei, delete the sensei directory, reinstall sensei and restore the config which is restore your custom sensei settings.

[the-mk]

will do a reinstall of sensei 0.8 too
looked at the /usr/local/sensei directory - mine was about 44 gigabytes - most of it in /usr/local/sensei/log/archive

[mb]

@Archanfel80, @the-mk,

With regard to archived logs, you can use the following commands to get rid of very old logs:

find /usr/local/sensei/log/active -type f -mtime +15d  | xargs rm -f {}\;
find /usr/local/sensei/log/archive -type f -mtime +15d  | xargs rm -f {}\;

Sensei health check system should have had this handled. Looks like a commit which did not end up in the release. Will integrate for 1.0.

For the elasticsearch data, along the way to 0.8, we changed the naming scheme for the indexes. This should be the reason why some indexes were not purged.

We'll also handle that with 1.0. For now, the workaround would be resetting reporting data (Sensei -> Configuration -> Reporting & Data) (be aware: this will delete all reporting history).

[mb]

@manjeet,

Currently, we're locking the os-sensei package. This is why OPNsense autoupdate do not update Sensei package. This was done for the period of integration to the OPNsense and for a more controlled software delivery. Lock will be removed shortly and Sensei will get updated along with other OPNsense packages.

Your feature request sounds cool; though we'll need to think a bit more on the correct implementation and also try to see how many other users would also be interested in this feature.

[bulmaro]

Sensei has detected swap was usage high (21 -- 13831872% usage) and has shut down Sensei services in order to prevent a network outage.

Any suggestions for my case?