Messages

Hi,

is there any update on it? Regarding the kernel options integration and to actually support all CPU Cores / threads from the CPU inside the system?

Hi @actionhect, yes. It might be sooner than we originally planned, since we have requests from school districts to handle 10 gig. Like netmap work, we need to work with OPNsense team on this. There are two kernel options (RSS, PCBGROUP) that need to be enabled in the kernel.

SVN team did some analysis on my router today and:

- confirmed that throughput indeed drops from >900 Mbps to ~250 Mbps when Sensei is on and active
- throughput goes back to >900 Mbps when Sensei is in bypass mode
- disabling the hyperthreading of firewall increased the throughput to ~350 Mbps

My device is one of the new-generation Protectli clones: https://www.aliexpress.com/item/4000803229693.html

i7 CPU with 32 GB ram and 500 GB mSATA

Will post an update once we progress more.

Is that a 7th generation proc ? I have an i5 8th and it seems that i got more physical CPUs.  :o

As far as i know, opnsense and sensei are not capable of using full cpu Power when you have multi-core or Multi threded CPUs. Only Single core with high MHz will get Performance, but not when there are multiple cores. Also Network stack is lacking Features like RSS and PCBGROUP kernel options with supoorting NIC Adapter  and so on to gain full Power out of your modern hardware...

https://forum.opnsense.org/index.php?topic=19420.0

Hi @ittk,

You can view details of the blocked session in Live Session Explorer by keeping the cursor on the blocked line, or Live Blocked Sessions Explorer.

True, but when only the blocked actions come from web controls action, the website warning page "blocked for reasons xyx" should be display to the user, as when you try to acesss adult content webpages. here when trying to access the opnsense forum, it's nothing displayed on the web browser (pahe just won't open), so it's having the same effect, like it is accessed when just the app control signatures have been applied.

Hi @ittk,

You can view details of the blocked session in Live Session Explorer by keeping the cursor on the blocked line, or Live Blocked Sessions Explorer.

The update time interval is in every hour. So you loaded the new version on Viewversions or with check updates?

With check Updates...

[1) But why is is there not in detail view which module an site was blocked, the reason for it? If its caused by web control rules (and which one) or by app controls, blocked app name rule in detail?]

Hi @ittk,

1- If it is enabled from the Configuration - Updates & Health, it updates automatically.
2- Web and AppDB are different. In High control, Blogs are also blocking and forum.opnsense.org category is "Blogs" in Web DB.

Hi,

thanks,

1) But why is is there not in detail view which module an site was blocked, the reason for it? If its caused by web control rules (and which one) or by app controls, blocked app name rule in detail?

In Live session view it just says (classified) as blocked application Online Utility --> OPNSENSE.
Or do i miss something to get the better view which modules (web or app controls and which exact rule of it) blocked it?

2) Here you go: but it's ticked all on:

Updates and Support   full help
Check For Updates Automatically:
Last Update Check: 11/07/2020 12:35   
Automatically update Databases And Threat Intelligence Data:
Last Updated: 12/10/2020 06:07

Whats the update und auto-install invervall? Maybe longer ranged, so i was before it run?

This morning i have to manually check for the APP DB update und installed and reloaded it by hand, as it was not yet installed.

Hi Tomsauy,

Just released a DB. Some Google Services and Apps updated and added with 1.6.20201209014859 AppDB. Looking forward your feedback.

Just updated the DB manually.

1. But why there is still no real auto-update for the DB?
2. This Opnsense Forum is completly blocked on report detected as Online Utility --> OPNSENSE, but within App Controls it is fully allowed! Web Controls are set to "High Control". Only lowering it to Moderate Control. Will workaround on it. So why does Report promtit as an APP being detected and blocked, when the issue maybe lies within the web control part and having selected the "High control" profile?

Its just after the full opnsense unit reboot, where this forum can be accessed for a while, but i guess when all sensei services and modules enignes are fully loaded, it will be blocked.

QUIC hosts drill down to 74.125.104.75 (Google) and 92.224.0.0/13 (Telefonica).

This Sounds thats sensei app detection only operates on layer 3/4 detection. This is not enough for the real App detection and control used in next Generation Firewalls -  as it must operate in upper Layer 5 up to 7. You must examine up to the Layer 7 to fully catch things like Youtube Video streams

In fact they always says it is 3,4 layer. We do not do ssl decryption as well. It's only SNI checking.

As you come form Fortinet you know it all. And yes, but this level (3 max layer 4) not not enough these days. So TLS up to v1.3 interception is required for the upper layer real app detection and also the more advanced malware scanning of network traffic which also totally lacks? Btw.: I don't see any real malware scan engine integration, yet now? Is it to come, soon?

And please, don't get it wrong, it's just all the good advice to further improve sensei ;)

@Tomsauy: I also had the strange issues that youtube app is not blocking while within live session view it was shown as blocked. i have fully reset of default the sensei plugin. and just enabled web control on highest level, and searched for app control (allwith youtube in it and also added quic). so far its seems to work, but currently i am not too convied it will keep so, as i have done this step already for the second time. And why cannot the APP DB really being updated with auto-update and install? Have not seen the feature, yet.

QUIC hosts drill down to 74.125.104.75 (Google) and 92.224.0.0/13 (Telefonica).

This Sounds thats sensei app detection only operates on layer 3/4 detection. This is not enough for the real App detection and control used in next Generation Firewalls -  as it must operate in upper Layer 5 up to 7. You must examine up to the Layer 7 to fully catch things like Youtube Video streams

While I see some Youtube again now (1.19G), there are still 2.91G QUIC "unaccounted for" over the last 24h. That's from my son's iPad, and all he does ATM is watch Youtube.
QUIC hosts drill down to 74.125.104.75 (Google) and 92.224.0.0/13 (Telefonica).
Telefonica is now owned by O2, which is the DSL provider used for the iPad's traffic. So either they have setup google caching infrastructure at the ISP, or, more likely, there is some peer to peer for Youtube going on now, because the net's description is:
descr:          ADSL Pool Customers
and one of the IPs reverse resolves as:
dynamic-092-226-002-016.92.226.pool.telefonica.de
which usually is a dynamic PPPoE IP here.

Yes, did the same observation - with Vodafone as Internet Access Provider when QUIC comes Info place...

So bad.... the result is the same [emoji20][emoji20]
I don't understand why...

Can you Post all your revent Version Infos from DB ...?

Hi,

Sensei warns when detected an update like in the attached screenshot1 then if it isn't installed manually, it updates automatically and shows info like attached screenshot2.

And is my Version the latest, see my above Posts.

Hi Sy
here is a pretty good example which came with the 11-26 update.

i disabled all unneeded VoIP apps. And allowed the transportation layer.
With the update the Telecom came by and was enabled.

So i had to open and recheck my settings with the update. And have to go through one by one for each group.
Here a filter or a different view would be cool to have.

thanks Sy!
cheers armin

I think there are two ways for implementation.

1. An special view / filter for new apps since last change view
2. The special color coding to optical identify freshly added apps since last view.

[So it seems there is not full AUTO-INSTALL which have to be initied after an auto-update detected for the signatures DBs, yet?]

Addition: My Testsystem is configured:

Updates and Support   

Check For Updates Automatically ON

But i have this Last Update Check: 11/07/2020 12:35   

Automatically update Databases And Threat Intelligence Data: ON

But i have this: Last Updated: 01/01/1970 01:33   

Enable Engine "Core File" Generation:    OFF

Cited from doc: https://docs.opnsense.org/vendor/sunnyvalley/sensei_install.html#updates-health-check

Check for Updates Automatically: Checks automatically for the updates and creates a notification on the Sensei "Status" page.

Automatically Update Databases and Threat Intelligence Data: Checks automatically for the updates and creates a notification on the Sensei "Status" page.

So it seems there is not full AUTO-INSTALL which have to be initied after an auto-update detected for the signatures DBs, yet? Any reasons for it, why this option seems to lack?

The installation of the new DB looks like to be triggered manually.

The update from the running DB is automatically.

@sy https://forum.opnsense.org/index.php?action=profile;u=23640

Yes, maybe broken? Anyone can confirm? Would be an big issue, not having the fully AUTO-UPDATE Feature working for such signatures DBs.

My state is this and have not clicked on Check Updates and Reload...

Engine Version:   1.6.1   
Last Update: 10/27/2020 19:07   
App & Rules DB Version:   1.6.20201021092213   
Last Update: 10/27/2020 19:07

Updating the app DB is really important.

Would be cool to have this as "auto" task in cron.

Isn't it working for you: "It updates automatically every hours and you can do it manually from Status page."