Sensei on OPNsense - Application based filtering

[mb]

A small note on how we do dns enrichment for ip addresses:

Engine doing the mapping realtime:

Engine keeps track of all dns transactions that it can see flowing over itself. When it detects an IP address resolution (either an A/AAAA/CNAME or PTR), packet engine caches the IP addresses and the corresponding fully qualified domain name.

All charts/tabular reports and live session reports display this cached hostname when you view the reports.

UI doing mapping during reports viewing:

This applies to live session reports only: When you view a live session report, while you're browsing over records, UI runs a background job to see if a particular record has its hostname resolved. If it detects an unresolved IP address, it runs a background query to resolve the IP address via the name server you've configured on Sensei -> Configuration -> Reporting and Data.

@the-mk, since daily reports are making use of realtime cached hostname resolutions, newly introduced feature will not have effect on them. 

@opnip, you should see them being resolved, while you're walking your mouse over them. Does that happen?

[opnip]

Thx for the hint. Yes, if i mouse over a IP address in "Live Sesssions Explorer" they would be resolved now.

[DeathWingMT]

Hi I would like some guidance on how to enable the web filtering feature. I have disabled the Adult site category for testing purposes and pointed my DNS to the OpnSense box running DNSMasq as the DNS server. Unfortunately, the adult site still loads. The manual does not provide any details on how to enable the service from a clients perspective or whether HTTPS is also filtered.

Note that I am using VLANs and have added the physical port as a sensei protected interface

[mucflyer]

Good day everyone. I do have issues with Sensei on my pfsense box. When starting Sensei Packet Engine, all traffic is gone. No ping to router, no internet, nothing.
OPNsense 19.7.4_1-amd64, Engine Version:1.0.3

[Ralf_s]

exclude devices?

Hi,

is it possible to bypass/exclude internal devices from scanning? i.e. there are streaming devices like Amazon FireStick  or Roon Rock that have issues with content.

I'm settimg all filters to allow - there are issues
I'm settimg the sensei engine in bypass mode - there are no issues

OpnSense are running on LANNER hardware with Intel C2558, 8GB RAM and server SSD.

best regards,

Ralf

[the-mk]

is it possible that the daily report mail is broken somehow since the upgrade to Sensei 1.0.3?
I've already checked the settings and performed to send a testmail (which arrived), as well as disabling and reenabling it did not help.
After the upgrade-process to Sensei 1.0.3 was successful one report mail arrived since then, but after that no more mails :(

[opnsenseuser]

@mb i thought atom c3558 is ok with sensei. but i get this (screenshot) if i try to configure sensei

[mb]

Hi @opnsenseuser,

It should be ok for you. You can just click on "Continue" and install Sensei. Your CPU looks almost good.

With 1.0.3, we've introduced this cpu benchmark, where we are measuring how powerful the cpu is. This was the first step to the upcoming 1.1 release where we'll have an alternative methodology for providiging Sensei for low-end devices like Deciso A10 / APU systems.

So the upcoming release will use Elasticsearch as the database if RAM is at least 4GB and more and CPU ubench score is higher than 300000.

If the amount of RAM is below 4GB and CPU is less powerful Sensei will use Mongodb as the database backend.

This way, we will be able to provide Sensei for low-end systems where cpu and RAM resources are limited.

We're only days away from creating the first BETA. If anyone interested to try out before the release, just PM me  ;)

[mb]

... Unfortunately, the adult site still loads. The manual does not provide any details on how to enable the service from a clients perspective or whether HTTPS is also filtered.

Note that I am using VLANs and have added the physical port as a sensei protected interface

Hi DeathWingMT,

VLANs should be ok. HTTPS/QUIC traffic is also filtered. We'll add this to the manual and make it more specific.

On the other hand, We'd like to diagnose as to what is going on during filtering in your case. First guess is loss of cloud connectivity.

I'll PM you, then we can have a look together.

[mb]

Good day everyone. I do have issues with Sensei on my pfsense box. When starting Sensei Packet Engine, all traffic is gone. No ping to router, no internet, nothing.
OPNsense 19.7.4_1-amd64, Engine Version:1.0.3

Hi @mucflyer, thanks for trying out Sensei. This looks like a netmap issue. Which ethernet adapter were you using?

[mb]

@the-mk, let's do a check, we'll update you.

@Ralf_s, whitelisting according to ip/vlan/user is available in the premium subscription.  The thing that you're not having any issues when in bypass mode make me thing we need to have a look at this.

I'll PM you.

[sol]

Unfortunately sensei chrashed after 3 to 5 days of usage:

Either is was high cpu usage or yesterday this happened:

Sensei has detected a problem during operation and has shut down Sensei services in order to prevent a network outage.

It is because we detected high SWAP (21 -- 13821280% usage)

I run sensei on OPNsense 19.7.4_1-amd64
Intel(R) Core(TM) i5-5250U CPU @ 1.60GHz (4 cores)
8 GB Ram
and also use proxy and ips
Connection is a 100/40 mbit line
and there are about 10 users

Restarting sensei works though, it just crashes after 3 - 5 days.

[sol]

And another question. How can I use sensei for my openvpn network. I cannot select it at the interface selection.

And local hostname resolution does not work for me or I'm not using the right configuration.
Opnsense runs unbound and dnscrypt proxy.

Which server do I have to use?
DNS server IP addresses to do reverse IP lookups:
127.0.0.1,192.168.1.1
is the current setup.

[Ralf_s]

@mb:
thank you for your answer. But the premium edition is to expensive for home use - only for the feature excluding IP addresses. I looking forward to your next releases. In the meantime, I'll use my Sophos XG home on an APU for transparent content/security filtering.

[mb]

Hi @Ralf_s,

Thanks for the feedback. Sunny Valley sales team is working on home use. Expect an announcement early 2020.