Sensei on OPNsense - Application based filtering

[mb]

Hi @shrdlu,

It was unfortunate that both OPNsense & Sensei got updated at the same time. Looks like while OPNsense was upgrading, we shipped 0.7.0-rc1. OPNsense update manager also updated Sensei, a case which we did not handle.

Sorry for the inconvenience. We rolled back 0.7.0-rc1.

A final fix will be out shortly.

For a workaround, I'll be contacting you. We'll try to recover the old configuration.

[mb]

Dear Sensei users,

0.7.0-rc1 upgrade is back.

A quick update on 0.7.0-rc1 upgrade:

If you encountered any Sensei issues while upgrading your OPNsense to 18.7.8, this was due an unhandled case in our package updater when the upgrade process is triggered from the OPNsense firmware updater, not from the Sensei Status Page. This is fixed now in the upcoming 0.7.0-rc1.

But the fix will be in effect starting from 0.7.0-rc1.

So, If you're on 0.7.0-beta1, and do NOT want to upgrade to 0.7.0-rc1 immediately we strongly recommend running the following command to avoid any issues with the OPNsense system updater.

pkg upgrade os-sensei-updater && pkg lock os-sensei

The command will upgrade your Sensei updater to the latest version and also put a lock on os-sensei package so that OPNsense package update utility will not try to update Sensei.

If you also want to upgrade  to 0.7.0-rc1: Navigate to Sensei -> Status -> Check Updates, and you'll be guided to upgrade to 0.7.0-rc1.

pS: 0.7.0-rc1 introduces several minor bug-fixes both on the updater and the UI. If we do not hear any issues, we'll hopefully release 0.7.0 in the coming week.

ppS: Thanks to increasing number of Sensei beta users, it looks like we need to increase bandwidth for Sensei Updates server (updates.sunnyvalley.io). Cool indeed  ;) This will be done in the following weeks. In the meantime, if you encounter slowdowns while installing / upgrading, we'd very much appreciate your understanding.

[samsonmcnulty]

Sounds fantastic! Good to see the adoption rate increasing at a healthy rate. I did encounter this error but it seems you are already aware of the issue:


***ERROR: Indices could not be created! Reporting may not work***



Is there a temp workaround? I assume uninstalling the package and reinstalling would work?

[mb]

Hi @samsonmcnulty

Yep, that would work.

Can you run the following commands. Basically it'll uninstall & install sensei

service eastpect onestop
service elasticsearch onestop
pkg delete elasticsearch5
pkg delete os-sensei
rm -rf /var/db/elasticsearch/nodes/*

You can also do that by selecting "Uninstall elasticsearch & Remove elasticsearch data" options while uninstalling from Web UI.

then to re-install it:

pkg install os-sensei

Sorry for the inconvenience.

One question: did that happen after you've done an OPNsense 18.7.8 upgrade? We're aware of this problem & hopefully fixed.

I wonder if there are other cases.

[dragon2611]

I'd like to try sensei but I suspect i'd run into problems with lack of RAM and also I have an opensense HA pair with one physical and one virtual (KVM) so I think i'd run into the KVM/VIRTIO issue.

I'm wondering if I'd be better off starting another virtual firewall and stuffing it in the traffic path for the machines i'd want to put behind sensei.

[mb]

Hi @dragon2611,

Good idea :) Let us know if you encounter any issues. On the virtual FW, you can use E1000 as network the adapter type:

https://guide.sunnyvalley.io/sensei/support/faq#can-i-run-sensei-on-a-virtualized-environment-like-proxmox-virtualbox-kvm

[Antaris]

Hi, Sunnyvalley.

The first hit and miss: try to block youtube used via google chrome...

[mb]

Hi @Antaris,

Thanks for reporting this.

It's because of QUIC: Google's new protocol suite, a replacement for TCP + TLS + HTTP/2. Chrome defaults to QUIC when you browse Google services. Other browsers use TCP so Sensei is be able to identify & block.

Sensei is able to identify QUIC, though its detailed protocol parser is under development. When we're done with it, it'll be able to identify protocols which are transported through QUIC. We hope to have it with 0.8.0 release.

[mb]

Dear Sensei users,

An update to: https://forum.opnsense.org/index.php?topic=9521.msg47488#msg47488 :

If you got stuck in Sensei Configuration Wizard,  here is a quick fix for you:

open /usr/local/sensei/scripts/installers/opnsense/18.1/sensei-init.sh file with an editor, and locate this part. It should be line 64.

if [ "$INDICES_COUNT" -lt 6 ]; then

Update this line to read like:

if [ "$INDICES_COUNT" -lt 5 ]; then

Save the file and re-run the configuration wizard.

0.7.0-rc2 will come with a more intelligent provisioning script which will try to diagnose any inconsistencies with the backend database and try to fix them automatically.

[mb]

Dear Sensei users,

An update to: https://forum.opnsense.org/index.php?topic=9521.msg47653#msg47653

We've decided to merge some of the code from the QUIC branch to 0.7.0.

Looks like we have a working app identification & filtering for applications running through QUIC :) e.g. Google Services + Chrome browser.

Update will be introduced with 0.7.0-rc2.

[Antaris]

Dear Sensei users,

An update to: https://forum.opnsense.org/index.php?topic=9521.msg47653#msg47653

We've decided to merge some of the code from the QUIC branch to 0.7.0.

Looks like we have a working app identification & filtering for applications running through QUIC :) e.g. Google Services + Chrome browser.

Update will be introduced with 0.7.0-rc2.

Thanks guys, looking forward to it. Can we hope for an optimisation to reduce hardware requirements, especially about RAM?

[mb]

Hi @Antaris,

Many thanks for bringing this into our attention. Looks like with 0.7.0-rc2, Sensei is one of the first in the industry to offer granular control for QUIC based applications.

Currently, big vendors are advising to completely block QUIC protocol, thus forcing browsers to fall back to TCP+TLS. This is slower.

As for memory requirements, actually yes. We're planning a limited reporting option, which will require way less memory than we require today. This will still provide reporting but most probably will lack some advanced features like Drill-down and per-connection details. Other than reporting, all features will be there.

[opnsenseuser]

when will sensei appear in opnsense as a plugin?
which hardware requirement is necessary?
Is the latest generation Atom processor (c3558) with 8 gb of ram sufficient for sensei?

thx

[Antaris]

There is an update Engine: 0.7.0-rc2, but when trying to update it, the system returns:  "No update is available
There are no updates available for you. You are using the latest version. " and stays on 0.7.0-rc1

[mb]

Hi @noname12123,

when will sensei appear in opnsense as a plugin?
which hardware requirement is necessary?
Is the latest generation Atom processor (c3558) with 8 gb of ram sufficient for sensei?

thx

We have a few small items left for the final OPNsense integration.  Then Sensei will be an OPNsense plugin which can installed from the OPNsense Plugins menu. If anything big does not come up, I guess we'll all be finished with them by the end of this month.

I'd expect that latest generation Atom would be ok. Might be a little bit slow to start Elasticsearch but when it warms up, it should be all fine. Crucial thing is RAM and 8GB is perfectly fine.