Messages

Hi @noname12123,

when will sensei appear in opnsense as a plugin?
which hardware requirement is necessary?
Is the latest generation Atom processor (c3558) with 8 gb of ram sufficient for sensei?

thx

We have a few small items left for the final OPNsense integration.  Then Sensei will be an OPNsense plugin which can installed from the OPNsense Plugins menu. If anything big does not come up, I guess we'll all be finished with them by the end of this month.

I'd expect that latest generation Atom would be ok. Might be a little bit slow to start Elasticsearch but when it warms up, it should be all fine. Crucial thing is RAM and 8GB is perfectly fine.

[one of the first]

Hi @Antaris,

Many thanks for bringing this into our attention. Looks like with 0.7.0-rc2, Sensei is one of the first in the industry to offer granular control for QUIC based applications.

Currently, big vendors are advising to completely block QUIC protocol, thus forcing browsers to fall back to TCP+TLS. This is slower.

As for memory requirements, actually yes. We're planning a limited reporting option, which will require way less memory than we require today. This will still provide reporting but most probably will lack some advanced features like Drill-down and per-connection details. Other than reporting, all features will be there.

Dear Sensei users,

An update to: https://forum.opnsense.org/index.php?topic=9521.msg47653#msg47653

We've decided to merge some of the code from the QUIC branch to 0.7.0.

Looks like we have a working app identification & filtering for applications running through QUIC :) e.g. Google Services + Chrome browser.

Update will be introduced with 0.7.0-rc2.

Hi @cgwork,

If you cannot see it reported, this means Sensei could not identify Google Drive. We tested it in our lab, same result. Looks like we need to improve detection for this protocol.

Once the detection is fixed, filtering will be automatically in place.

Stay tuned for updates on this.

Dear Sensei users,

An update to: https://forum.opnsense.org/index.php?topic=9521.msg47488#msg47488 :

If you got stuck in Sensei Configuration Wizard,  here is a quick fix for you:

open /usr/local/sensei/scripts/installers/opnsense/18.1/sensei-init.sh file with an editor, and locate this part. It should be line 64.

if [ "$INDICES_COUNT" -lt 6 ]; then

Update this line to read like:

if [ "$INDICES_COUNT" -lt 5 ]; then

Save the file and re-run the configuration wizard.

0.7.0-rc2 will come with a more intelligent provisioning script which will try to diagnose any inconsistencies with the backend database and try to fix them automatically.

Hi @cgwork,

Are you using Chrome? If so, the answer could be:

https://forum.opnsense.org/index.php?topic=9521.msg47653#msg47653

If not, can you check whether Sensei detected any Google Drive (Sensei -> Reports -> Connections) ?

Hi @Antaris,

Thanks for reporting this.

It's because of QUIC: Google's new protocol suite, a replacement for TCP + TLS + HTTP/2. Chrome defaults to QUIC when you browse Google services. Other browsers use TCP so Sensei is be able to identify & block.

Sensei is able to identify QUIC, though its detailed protocol parser is under development. When we're done with it, it'll be able to identify protocols which are transported through QUIC. We hope to have it with 0.8.0 release.

Hi @dragon2611,

Good idea :) Let us know if you encounter any issues. On the virtual FW, you can use E1000 as network the adapter type:

https://guide.sunnyvalley.io/sensei/support/faq#can-i-run-sensei-on-a-virtualized-environment-like-proxmox-virtualbox-kvm

Hi @samsonmcnulty

Yep, that would work.

Can you run the following commands. Basically it'll uninstall & install sensei

service eastpect onestop
service elasticsearch onestop
pkg delete elasticsearch5
pkg delete os-sensei
rm -rf /var/db/elasticsearch/nodes/*

You can also do that by selecting "Uninstall elasticsearch & Remove elasticsearch data" options while uninstalling from Web UI.

then to re-install it:

pkg install os-sensei

Sorry for the inconvenience.

One question: did that happen after you've done an OPNsense 18.7.8 upgrade? We're aware of this problem & hopefully fixed.

I wonder if there are other cases.

Dear Sensei users,

0.7.0-rc1 upgrade is back.

A quick update on 0.7.0-rc1 upgrade:

If you encountered any Sensei issues while upgrading your OPNsense to 18.7.8, this was due an unhandled case in our package updater when the upgrade process is triggered from the OPNsense firmware updater, not from the Sensei Status Page. This is fixed now in the upcoming 0.7.0-rc1.

But the fix will be in effect starting from 0.7.0-rc1.

So, If you're on 0.7.0-beta1, and do NOT want to upgrade to 0.7.0-rc1 immediately we strongly recommend running the following command to avoid any issues with the OPNsense system updater.

pkg upgrade os-sensei-updater && pkg lock os-sensei

The command will upgrade your Sensei updater to the latest version and also put a lock on os-sensei package so that OPNsense package update utility will not try to update Sensei.

If you also want to upgrade  to 0.7.0-rc1: Navigate to Sensei -> Status -> Check Updates, and you'll be guided to upgrade to 0.7.0-rc1.

pS: 0.7.0-rc1 introduces several minor bug-fixes both on the updater and the UI. If we do not hear any issues, we'll hopefully release 0.7.0 in the coming week.

ppS: Thanks to increasing number of Sensei beta users, it looks like we need to increase bandwidth for Sensei Updates server (updates.sunnyvalley.io). Cool indeed  ;) This will be done in the following weeks. In the meantime, if you encounter slowdowns while installing / upgrading, we'd very much appreciate your understanding.

Hi @shrdlu,

It was unfortunate that both OPNsense & Sensei got updated at the same time. Looks like while OPNsense was upgrading, we shipped 0.7.0-rc1. OPNsense update manager also updated Sensei, a case which we did not handle.

Sorry for the inconvenience. We rolled back 0.7.0-rc1.

A final fix will be out shortly.

For a workaround, I'll be contacting you. We'll try to recover the old configuration.

Hi @bobbythomas,

/tmp/sensei_update.progress should have more detail regarding the update process. 36MB download shouldn't take that long.

We rolled back rc1 update in case there is something we miss with the update process.

Hi @johjoh,

Yes :)

A big portion of the resource requirement come from the Reporting engine (Elasticsearch). The core packet engine has been tested to run on low resource systems: e.g. Celeron  < 1GB RAM.

A roadmap feature - remote reporting - allows to run packet engine on the firewall itself, and reporting on another more powerful server.

Hi @maekar ,

This is the current field-tested maximum. Software arranges several tunables (e.g. cache sizes, connection table sizes etc.) according to the user size.

Current focus is to make the software super stable for SME use cases (which generally means user populations below 1000)

Looking forward, engine is able to scale to hardware resources, which makes it possible to secure thousands of users.

Hope this answers your question.

Hi @manjeet,

Thank you for sharing your experience with Sensei. We very much appreciate that. Find the answer below:

Is there any live view to know which IPs are using the most bandwidth and then drill down to specific IP to check what sites - services it is using and which site - service is consuming the most

Yep. Navigate to Sensei -> Reports -> Connections. Look for the Chart named Top Local / Remote Hosts. But make sure to select the reporting criteria as "Volume" from the upper right hand corner of the reports page. Default is by sessions. You can do "Session based", "Packet based" or "Volume based" reporting.

When you left click on any IP, a submenu appears. Click "Drill-down" and all reports will be automagically filtered according to this IP address you've selected.

I do not know why but when i check the "Table of Local / Remote hosts, it shows bytes in / out which is very low as compared to ntopng. and i have avg 25-30 devices running all the time out of 50 devices but it only show few so how can i list all of them.

My guess is that you might be viewing the "Session" reports. Make sure you've selected "Volume" as the reporting criteria.

All devices currently active should be listed though. My guess would be that you might be viewing reports for the last 15 minutes. Make sure you've selected a longer time frame from the right hand corner.

Is there any way to get all the web history of a user or users ?

Yes. You can do that from the Web / TLS reports. You have the drill-down capability for every report type.

Is there any way to bind names to IP (local / LAN IP) as squid does in web proxy ?

Actually, packet engine automatically maps DNS names to IP addresses if it can find a matching DNS transaction. Soon there will be Active Directory / LDAP integration which you'll be able to see the user / group names.

It filter web traffic and works as transparent web proxy so is there any way to use it as cache server as well or if you are planning for it in future.

In theory, packet engine is capable of doing that. But we chose to focus on complementing features that are currently not existing. Squid is a great caching proxy. Indeed caching is its original reason of existence. That being said, Sensei roadmap does not have "caching" as a feature.

Many thanks for reporting your experience with us.