Sensei on OPNsense - Application based filtering

[nikkon]

yes. this is before i executed both scripts
it's not solved.

[mb]

Hi @nikkon, understood. Let's do some more debugging together. I'll contact you.

[Antaris]

Very often i see remote hosts in local table and vice versa. Is something wrong with my setups?
And sometimes i see comunication between two local ip addresses and one of them is marked as remote...

[mb]

Hi @Antaris,

Do you have multiple interfaces configured for Sensei? Are these IP addresses multicast / broadcast addresses?

[Antaris]

I have only LAN selected in Sensei with only one IP and no VLANs on it. The adresses are known internal hosts. Not broadcast or net addresses.

[mb]

Dear Sensei & OPNsense users,

Happy new year to all. Here is a humble new year present from Sensei team.

We're happy to announce the availability of Sensei 0.7.0 release. It was ready since last Friday, but we wanted to make sure everyone had a calm Christmas holiday, spending time with friends and family instead of doing Sensei deployments
 
This is the full list of features that this release brings (from 0.6.x):

1. 350+ new applications identified.
2. Google applications browsed via Chrome are now being identified (QUIC over UDP protocol support).
3. Mobile browser compatibility: you can view reports from your mobile browser
4. Reports enhancement: Data retirement option introduced. With this option you can define how long to keep your reports (days)
5. Reports enhancement: Option to erase all reporting data
6. Reports enhancement: Drill-down in Security reports is now available
7. Reports enhancement: Daily executive reports. Selected reports delivered via a daily e-mail.
8. You can easily add block/allow rules within Session Explorer based on Application and Application Category or SNI / hostname
9. User's Manual in English.
10. More deployment options for Home and Large scale users
11. Changelog between updates
12. Fixed Rebellion Theme compatibility issues.
13. Better Cloud Nodes availability
14. Better & smoother updates
15. We speak your language now, we added i18n support to match your OPNsense UI language. English and German are there for now, more coming soon.
16. Removed some large dependencies in preparation for embedded devices & PIE (Position Independent Executable) support.More performance & stability improvements.

To update your installation, simply navigate to Sensei -> Status and you should see 0.7.0 update being reported and an option to install it. If you do not see the update notification, just click "Check for updates" and you'll be guided through the update process.
 
A quick note: Although this is marked "release", Sensei is still under BETA development. We strongly advise to test the software on one of your test-beds to see if it fits your requirements. When we finally release Sensei 1.0, the BETA program will cease and the software will be publicly available for all users. We expect to release Sensei 1.0 in Q1 2019.
 
If you find any issues or you want to reach out for comments and feedback, please do not hesitate to contact us through sensei -at- sunnyvalley.io or through this forum thread.
 
Happy new year to all

Sensei team

[manjeet]

Thanks for the update @MB.

For Table Reports update is working as expected.

As @Antaris mentioned, i also see remote host in local table but no local host in remote table except OPNsense LAN IP which i think, in one way, is not an issue because firewall itself generates traffic for interface access etc..

[Antaris]

Also thanks from me for the update.

"12. Fixed Rebellion Theme compatibility issues."

In session details the headers of the columns are still with white text on white background:

https://www.dropbox.com/s/0v72em2bch0rk0q/Reb.jpg?dl=0

[donatom3]

Can't tell if this is a new issue or not as I only installed of of .7.0-rc3. When the packet engine is running unbound overrides are being ignored.

My nslookup results show "UnKnown" in the server spot and are forwarding my overrides to public servers.
As soon as I stop packet engine this works again.
I was able to add my root domain to the "local domain to override" section and it fixed that one issue there but I have overrides for other hosts. Am I missing a setting where Sensei is overriding DNS?

[mb]

Thanks for the update @MB.

For Table Reports update is working as expected.

As @Antaris mentioned, i also see remote host in local table but no local host in remote table except OPNsense LAN IP which i think, in one way, is not an issue because firewall itself generates traffic for interface access etc..

Hi @manjeet, you're very welcome. Can you share with me a screenshot of  the remote hosts table (you know my email). Would like to see how they look like. Normally you should only see local hosts behind the firewall there.

[mb]

Hi @Antaris,

You're all welcome & thx for the pointer. We'll fix it.

[mb]

Hi @donatom3,

Actually this is an expected behavior. We're utilizing DNS override for Web Reputation & Threat Intel. Since DNS occurs before the actual connection attempt, we gather prior threat intelligence & reputation about the remote IP & host.

For a quick workaround you can disable Cloud Reputation & Web Categorization from Sensei -> Configuration. Then you'll still have reputation data for the top 1Million domains from the local database, but not for +140M .

We're exploring ideas to do this in parallel. This way you'll still be able to do your DNS through your DNS server and Sensei will do a parallel query for its intelligence.

[donatom3]

@mb this is good to know.
So if I'm in an environment where I'm using Windows domain controllers for DNS to get the full effect of Sensei would I need to have the opnsense router be the DNS forwarder?

Also does this mean if I just hand out public DNS servers via DNS am I not getting the full advantage of Sensei?

P.S. I do want to add that I am liking Sensei so far.
I am still able to download at 1gbps on my i5-5250u but thinking of picking up a box that has an i5-6500.

[mb]

Hi @donatom3,

For application control, dns does not play any role there, so you'll be utilizing Sensei at its full potential in any ways.

For security & web filtering, yes, you'll lose some data there, provided that you do not enable Web Reputation & Cloud Threat Intel, which requires DNS override.

If your DNS transactions are traversing through an interface which is protected by Sensei & you have Cloud enabled, dns override will be in place (like unbound). Because we are way earlier in the process, we'll also override unbound.

That being said, if you place the DC in the same broadcast domain with your clients, they'll be utilizing your DC (this way they do not traverse through Sensei to reach the DNS server). -- Local DNS server will still be subject to the same behavior if its traffic passes through Sensei. -- Here you have your local queries untouched.

We've been asked (by some more users) of a possibility of using both Sensei Cloud database & local dns servers. We're evaluating methods to do it. Most probably we'll be back into this in the second or third quarter of 2019.

Very happy to hear that you've attained gigabit speeds and happy with the software

[jinn]

I tested sensei last week. after I activated it, however, access to the internet was barely possible (eg google was not available at all). since it was a productive system, I deactivated sensei for now and did not continue to use it. Nevertheless, I would like to know why it was located, how should I proceed best for analysis here?

In addition, I wanted to ask whether it is even possible, what I want to achieve: I would like an evaluation for special services (social media, online gaming, ...). Is sensei able to give me an evaluation of how much time / data was used for special services?