Sensei on OPNsense - Application based filtering

[jjanzz]

We're currently working with OPNsense for the integration. As a part of this process, OPNsense started providing some of our dependencies. We'll update our installer and package to be compatible with the latest changes on OPNsense package repository, soon.

Thank you for the reply. No problem; I'll gladly help you test it out as soon as it's possible :)

[mb]

@jjanz and community,

Elasticsearch5 was added to OPNsense packages as part of the 18.7.5 update. There was a problem in the FreeBSD elasticsearch package builds which was inherited by the OPNsense build system.

Because elasticsearch was problematic, Sensei installations were failing.

Today we fixed the problem. In the meantime, OPNsense will be removing the package from its repository in the upcoming release.

Starting 18.7.6, elasticsearch will be provided by Sunny Valley Package repository.

Long story short: We're resuming Sensei downloads. You can now download and install new Sensei version, which is 0.7.0-beta1 as of now.

[mb]

Hello all,

As part of 0.7 release effort, we've launched Sensei Users' Manual & Documentation.

Please find it here:

https://guide.sunnyvalley.io/sensei/

[wordsmith]

This plugin looks pretty interesting and I'd like to give you some non-technical feedback to consider. But first a question: will Sensei ever be open source?
See, the reason I ask is because to me it seems there is some confusing communication going on. I'm sure, some of it is non-intentional like:

For now I'm happy to tell that community edition for OPNsense will always be there and forever free.

"For now" and "always" don't work well together. Basically, now you're saying that this will always be the case, but later you might change your mind to "it isn't free anymore". I suspect that this was unintentional, but I just wanted to get it out of the way.

What rubs me a bit the wrong way is that the community edition is free, but not open source. According to your FAQ:

The Packet Engine coded in C++, and its source code is not open.

I think the reason there are community editions in the software space is precisely to indicate that a company/developer wants to build a trust model with others and, as a result, gives them the recipe so that they can build a community around it together. In short, it isn't about getting something for free i.e. without having to pay, but to build trust.

Now, where your approach to marketing proofs to be rather problematic is with statements like this:

Empower your open source firewall with Next Generation features.

If you plan to keep parts of Sensei closed source, I'd suggest you'd drop the "open source" in your marketing, because it's confusing at best, misleading at worst. Next, as long Sensei isn't open source, I'd also reconsider the use of "community edition": this is a rather well known way to describe the non-commercial version of a product that isn't just for the community, but also by the community. If the community doesn't have access to the code, it's not a community edition, it's a free edition.

The FLOSS community already suffers from a huge labeling problem (ever tried to explain to a non-technical user the difference between Free Software and Freeware?) so let's not muddy the waters even more.

I don't know about your business model, but for people who really care about open source it's not about getting stuff for free, it's to be able to verify the claims of company such as yours and, of course, to build a community around a solution that can be build by like-minded people without restrictions regarding code access.

Of course, at the end of the day there's always the pragmatic side to consider and there will probably be a lot of users who are perfectly fine to run proprietary software on their open source OS, but for people like me who decided to use an open source solution not because it is free of charge, but precisely because it's source code is available, Sensei won't be the solution we're looking for.

Now, with all that being said, I still appreciate your efforts.

[mb]

Hi @wordsmith,

Many thanks for taking the time and provide this valuable feedback. Now we become aware of a communication problem.

To clarify things:

• Either Community or Enterprise Edition, Sensei core packet engine is not open source. Speaking of today,  there are no plans to make it open source. We're like Oracle running on Linux.
• Community Edition (or may be we should call it freemium edition) is and will be free of charge for the open source community. - Free as in "free beer"  ;)

As you've correctly pointed out, if there is any misunderstanding, it's unintentional. Your comments shed a lot of light as to what needs to be adjusted in the messaging. We'll be working on that.

Taking this chance, I'd like to give a little bit of background why we started with "open source firewalls".

As Sensei team, we believe that we've created a powerful packet processing technology. We believe that better packet visibility means better decision making. Better decision making means better success rates in detecting malign traffic.

Sensei is the first of two products that we're going to create for a large market.

We hope to make Sensei available for any network security equipment / product which needs application classification & web security features. L3-L4 firewalls, UTMs all fall into this category.

The thing we started with open source firewall space is that, it was a request by an MSP who was deploying open source firewalls onto customers and providing support services. Very happy with their current firewalls, they needed several features that we could provide. We quickly did an integration and voila! The resulting solution (OPNsense + Sensei) was found to be better than many of the current players in the UTM market.

This sparked a light for us. Why not deliver the product as a plugin instead of yet another full-blown firewall appliance? It'd be cost effective for us and we would than be able to relay this cost advantage for the benefit of our prospective users.

In this regard, open source firewalls is a delivery channel for us, though it's not the complete target market. Via this initial channel, we learn very much from our users and improve Sensei. You can't believe how much Sensei improved from the day we announced first beta up until this day. Then of course, we are looking for market visibility. It's great to see people loving the solution and spreading the word.

A free of charge Sensei edition (maybe we should call this freemium edition) is a way of our giving back to the OPNsense community.

Having founded a local open source community (enderunix.org) and published some open source tools, I truly understand, appreciate and respect your stance.

Though we cannot make Sensei fully open source, I think the best we can do right now is to communicate what Sensei is and what it is not in a straight and open way. This way people would know what they will have and what they won't; and will make an informed decision about using / not using it.

It's somewhat hard to figure out a way to communicate people that the current product is for "open source firewalls" without using the words "open source". Because marketing wise, we would like to be as precise as possible so that people would know what it is for.

However I also see that it's creating confusion. We'll spend more time on this. I'd also like to consult you if you wouldn't mind.

Again, many thanks for bringing this up to our attention.

[mb]

Dear Sensei users,

0.7.0-beta1 update is out for those who are on 0.6.x releases:

https://www.sunnyvalley.io/blog/0-7-beta1-update-available-for-0-6-x-users

0.7 Beta1 comes with the following functionality:

1. New Report - Blocked Connections Sessions Explorer and drill-down reporting
2. Reports enhancement: Daily executive reports. Selected reports delivered via a daily
    e-mail.
3. Customizable Landing Page for Blocked connections
4. Reports data retiring: disk space consumed by Elastic Search (Reports) is now
    configurable
5. Release Changelog is now displayed during Sensei updates
6. Shortcut to add Block/Allow rules based on fields (IP Address, Application, App
    Category etc.) via Session Explorer Reports. 
7. 350+ new applications identified.
8. Documentation: Sensei Users' Manual
9. Sensei speaks your language now, we added i18n support to match your OPNsense
    UI language. English & German are the two for now, more coming soon.
10. More performance & stability improvements


If you've downloaded & installed Sensei later than October 15, you should already be using 0.7.0 beta1. This is an update package for older versions.

[shrdlu]

Not sure if this is the right place to post this, so if I am wrong please redirect me.

I have noticed with Sensei (BTW, it is working fine) that when I run a health audit in OPNsense I get the following (see attached screenshot) checksum mismatch for the nodes.csv file and was curious if this is normal or something is wrong.  Things appear to work fine and no matter what Cloud Threat Intel selections I make (not sure that is related but it might be) I get the mismatch and the Cloud Threat Intel is working fine regardless, or at least shows up and running.

And, on another note, in terms of processing when do the Sensei components process information in terms of order?  For example, I use the web proxy (squid) in OPNsense and was curious if Sensei process the packets before the proxy or after or somehow during, or frankly something completely different if I am misunderstanding the order of operations.

Thanks in advance.

[mb]

Hi @shrdlu,

You're in the correct place :) We're receiving feedback & comments and help requests here. You can also shoot a ticket if you think you've found an issue with the software:

https://gitlab.com/svn-community/opnsense-sensei-plugin/issues

The thing with Node.csv is not an issue. Web UI updates the contents of this file with the best servers available. I guess this creates a mismatch with the OPNsense File Integrity Checker. We'll handle that.

With regard to processing order: Sensei receives packets while they traverse from Network Adapter to the FreeBSD networking stack; which means it receives them before Squid and even before L3/L4 Filtering.

You're all welcome, and thanks for sharing your experience.

[theq86]

I Installed sensei. When I was on the dashboard to configure the protected interfaces only my 2 vpn interfaces show up. Not WAN, not LAN, nor any other interface on my firewall.

current version as of writing (0.7.0-beta1)

[mimugmail]

Do you have IPS enabled on LAN or WAN?

[theq86]

Nope. Neither

[mb]

Hi @nasq,

Any chances your LAN interface is virtio?

https://guide.sunnyvalley.io/sensei/support/faq#no-ethernet-interface-is-being-shown-in-the-interface-configuration

As quick workaround, select Intel E1000 as the adapter type.

As the final solution we're sponsoring a development which will ship the latest upstream netmap code into FreeBSD.

This will also fix lots of issues that you might be encountering with Suricata as well.

https://svnweb.freebsd.org/base?view=revision&revision=340436

It's us. Commit is done to HEAD, will be MFC'd to 11-STABLE in the following timeframe.

[mimugmail]

https://svnweb.freebsd.org/base?view=revision&revision=340436

It's us. Commit is done to HEAD, will be MFC'd to 11-STABLE in the following timeframe.

Really nice contribution Murat, thanks! :)

[mb]

Hi @mimugmail,

Our pleasure. All welcome :) Super excited to see the changes land in 19.1.

[franco]

r340436 is indeed very nice. mb, please push these into my mailbox or open a src.git ticket for swift inclusion. we need the MFC for stable/11 to be committed first though.

for the csv, it's considered bad style to manipulate files shipped with the package. for that reason FreeBSD has the "sample" trick which creates a copy of the file and only checks in the unmodified file (suffix ".sample"). We use it in core in some places, too. Plugins don't support it yet, but they should eventually.


Cheers,
Franco