Sensei on OPNsense - Application based filtering

[Marcel_75]

Hi,

thx for the update, also checked the behaviour again – but this time not with my "Mac-Warez" blocking sites, but with an own whitelisting area:

Sensei | Web Controls | User Defined Categories

"Whitelisted-Sites"

I've added all these sites to have the Ookla Speedcheck from https://www.speedtest.net/ working (not sure if all of them are needed for the Speedtest, but with the help of the uMatrix-plugin I could see they are accessed when you open speedtest.net)

1    *.cdnst.net    
2    *.cronon.net    
3    *.gtt.net    
4    *.ooklaserver.net    
5    *.speedtest.net    
6    *.wittenberg-net.de

But again it was only working like expected after a complete restart of my OPNsense … 

Not a big issue for me, as it's fine if it's working as expected after a restart, but of course it would be nicer if these filters will be active when you change them without an extra restart … 

PS: Strange, it worked after the restart – but as I was posting this, now it's not working again, Firefox can't open the site.

So it was working for some minutes after the restart but is now blocked again by Sensei? (if I switch sensei off, it's working fine … tested this of course)

[opnsenseuser]

@opnsenseuser, we'll be revisiting css/jscript codes.

Thx. If you need help just ask!

@mb one more thing

for popup´s you also need to use the original opnsense classes. so it´s easier for the sensei plugin to work with all themes. see my screenshots! thx

[mb]

Hi @ctr,

Thanks for the detailed feedback and trying out Sensei.

If you do not have a preference, we suggest you have the main interface for the VLANs. When you configure the main interface (e.g. ix1 in your case), it will be effective for all of the VLANs on this interface.

Because of a netmap-bug we deliberately prevent both parent/child interfaces configured at the same time.

Can we have a look at your installation? Non-routable IP addresses shouldn't be enriched with GeoIP data.

[ctr]

I tried to add ix1 (on it's own). This is the situation where I have significant packet loss as Sensei is enabled.

[mb]

1    *.cdnst.net    
2    *.cronon.net    
So it was working for some minutes after the restart but is now blocked again by Sensei? (if I switch sensei off, it's working fine … tested this of course)

Hi Marcel,

Thanks for the update. Can you try them without the leading "*." characters? That might be the thing. cdnst.net/cronon.net should match for all subdomains.

If it's not working, just PM me so that we can have a look together. 

[mb]

@ctr, ah, this is a bummer. I'll PM you so that we have a look at it together.

[mb]

for popup´s you also need to use the original opnsense classes. so it´s easier for the sensei plugin to work with all themes. see my screenshots! thx

Yep, we'll need a work there.

[samsonmcnulty]

@mb any chance you'll provide a lifetime pricing model that would work to provide some of the more advanced features to home labbers with a small number of users instead of the monthly subscription model?

[mb]

Hi @samsonmcnulty, thanks for your interest. Can't promise for a lifetime licensing, but we'll make sure we provide a "home" edition, which will have a relevant affordable pricing as soon as we have some progress with the current offering.

[seitzbg]

Trying to install Sensei 1.0 on OPNsense 19.7.2 and it will not let me pick the WAN interface to protect.  Any ideas?

[xpendable]

Hello,

I recently installed Sensei in my home environment, here is my experience with it so far and thoughts/requests for the product.

Sensei does not seem to install properly on LibreSSL with the current fix (v1.0.2) as os-sunnyvally plugin is unavailable and does not get installed. Also after selecting the LAN interface, even though it is in the selected list in the configuration tab... Sensei packet engine fails to start indicating that you must select at least 1 interface and no Cloud nodes are listed in the status page as well as no selected interfaces. After switching back to OpenSSL, installing the os-sunnyvalley plugin and doing a factory reset in Sensei, I was able add the LAN interface and Sensei then works as expected.

While talking about interfaces, I am unable to add my VPN (WireGuard) interface to Sensei successfully. Once added the status page says that there are no interfaces selected and the cloud nodes are also no longer listed, however the Sensei packet engine continues to run. I created an interface (OPT1) and assigned (wg0) network port to it with no additional settings, and this is the interface that I added to Sensei with no success. Are there plans to add support for assigning a WireGuard VPN interface within Sensei?

So far I am quite happy with Sensei's overall performance and the features that it provides, but I was hoping that it would completely replace my previous suricata/pihole setup that I had before for the LAN with one of the main functions being to block ads network wide. However I have noticed that the current ad blocking provided by Sensei does not appear to be quite as good when compared to the pihole, but it's hard to say for sure. Also since the VPN interface is currently unprotected, no VPN clients receive the benefits of Sensei as I did before with the pihole setup.

I did see the announcement of supporting community filter lists in a future update, so that will more then likely provide more ad blocking coverage along with providing additional block lists for other categories which will be great for the community edition.

Some nice things that I would like to see change would be to make the health checks based locally and to have an option to provide statistics back to Sunny Valley. I don't see why these checks need to be run/verified on a remote Nagios server. I believe most cpu/memory/disk checks... etc. can be run on the local server via either a local script and/or using Monit for these checks and alerts.

I'm sure this is probably in the works, but adding a widget for Sensei Status would be great to be able to have a quick look available right from the OPNsense dashboard.

[mb]

Trying to install Sensei 1.0 on OPNsense 19.7.2 and it will not let me pick the WAN interface to protect.  Any ideas?

Hi @seitzbg, thanks for trying out Sensei.

We filter out the WAN interface. Reason is Sensei grabs the packets after the network stack is done with them in the outbound packet flow.

In the practical sense, in case of NAT (nearly all of the use cases), when we deploy on WAN interface, we loose local IP address information.

[mb]

Hi @xpendable,

Many thanks for trying out Sensei and providing a detailed review. This is one of the things we love for making Sensei available in an open source community. We receive very quality feedback. I strongly believe quality feedback helps build great products.

Sensei does not seem to install properly on LibreSSL with the current fix (v1.0.2) as os-sunnyvally plugin is unavailable and does not get installed....

We're building a separate repo for LibreSSL. As a workaround for now, 1.0.2 can install onto a LibreSSL deployment with the old method where we do not configure our repository with the help of a package.

Starting with 1.0.2, this workaround should actually be solving this. I'm guessing that you might have tried a bit earlier before we updated the getsensei script.

While talking about interfaces, I am unable to add my VPN (WireGuard) interface to Sensei successfully.

Can you try this command to see if you are reported any errors and packet transmission is ok during the test. Make sure sensei and suricata is not using this interface during your test.

Code: [Select]

# ifconfig wg0 up -txcsum -rxcsum -tso4 -tso6 -lro -txcsum6 -rxcsum6
# /usr/local/sensei/bin/nmbridge -i netmap:wg0 -i netmap:wg0^
If you experience any problems here, then the issue here is netmap,  the I/O subsystem that we are utilizing to access the raw packets off the wire, and it does not play well with some interfaces. Last year, we sponsored a development effort to add support for virtio and vmx interfaces and this also came along with some reliability fixes.

Budget permitting, this year, we'll sponsor another development effort which will just focus on interface support and reliability fixes.

When it's done, I expect that more issues should have been addressed, including better interface support.

So far I am quite happy with Sensei's overall performance and the features that it provides, but I was hoping that it would completely replace my previous suricata/pihole setup that I had before for the LAN with one of the main functions being to block ads network wide.

We'll do a more thorough check with a special emphasis on ad blocking.

I did see the announcement of supporting community filter lists in a future update, so that will more then likely provide more ad blocking coverage along with providing additional block lists for other categories which will be great for the community edition.

Yep, we're looking forward to delivering this asap.

Some nice things that I would like to see change would be to make the health checks based locally and to have an option to provide statistics back to Sunny Valley. I don't see why these checks need to be run/verified on a remote Nagios server. I believe most cpu/memory/disk checks... etc. can be run on the local server via either a local script and/or using Monit for these checks and alerts.

During beta period, these statistics have proven to be lighthouses for us in spotting some issues. We have an open development item to make this optional.

I'm sure this is probably in the works, but adding a widget for Sensei Status would be great to be able to have a quick look available right from the OPNsense dashboard.

Yes, along with a more dynamic Sensei dashboard, this is in the works.

Again, thanks for taking the time to provide this detailed feedback.

[xpendable]

Hi mb,

Per my main issue below, I disabled suricata, left the VPN unassigned in Sensei and tried to run the below commands. However the ifconfig command gave me an error straight away saying "ifconfig: -txcsum: Invalid argument".

Taking away any option such as -txcsum to start with -rxcsum results in the same error but on the next switch, in this case -rxcsum

So I'm guessing this is a netmap issue? fyi, I have OPNsense running in a VM on ESXi using the vmxnet3 vNIC. I have also enabled the following tunable (vmxnet3.netmap_native = 1) as I believe netmap was updated in v19.7 with support for this option.

Hopefully this can be resolved at some point as I would really like to protect the VPN interface using Sensei. Thanks for getting back to me and I look forward to future updates, especially the community filter lists

UPDATE:
So I decided to do the nmbridge test even though the offload settings could not be disabled via the ifconfig command. See attached for the results, I did one test with an active VPN connection and one with no VPN connection.

Can you try this command to see if you are reported any errors and packet transmission is ok during the test. Make sure sensei and suricata is not using this interface during your test.

Code: [Select]

# ifconfig wg0 up -txcsum -rxcsum -tso4 -tso6 -lro -txcsum6 -rxcsum6
# /usr/local/sensei/bin/nmbridge -i netmap:wg0 -i netmap:wg0^
If you experience any problems here, then the issue here is netmap,  the I/O subsystem that we are utilizing to access the raw packets off the wire, and it does not play well with some interfaces. Last year, we sponsored a development effort to add support for virtio and vmx interfaces and this also came along with some reliability fixes.

Budget permitting, this year, we'll sponsor another development effort which will just focus on interface support and reliability fixes.

When it's done, I expect that more issues should have been addressed, including better interface support.

[opnsenseuser]

@mb I programmed the sidebar and I have tried sensei now. but note that something in the menu structure must be different because the automatic closing of the previous menu item only works if sensei is not activated as an active menu. To make it easier I attached a screenshot. I think there must be something different in the menu structure.

@mb menu problem solved!!
working on the "css code" fixes for sensei now!! this will come later this week!!

https://github.com/opnsense/core/pull/3653