OPNsense Forum

English Forums => Zenarmor (Sensei) => Topic started by: Tomsauy on November 13, 2020, 11:49:56 pm

Title: Block YouTube App
Post by: Tomsauy on November 13, 2020, 11:49:56 pm
Hi all

I am currently trying to test Sensei with some guys inside a kind of association 😊
For our test we are trying to block YouTube using OPNSense + Sensei free edition (many of us want to manage streaming consumption at home... and YouTube is a good example for us)

Under App Control we have disable anything related to YouTube ! And it works great when we want to access YouTube inside Chrome or Firefox 👍
But when I launch YouTube app inside Android mobile phone I am able to start video and play contents... 😞
Did we miss something ?

Can someone try to test and tell us the results ?

Thanks for your help !! 😊
Title: Re: Block YouTube App
Post by: bunchofreeds on November 15, 2020, 03:11:54 am
How are you selecting your devices to be managed.
The reason I ask is that Android devices randomised their MAC address so will likely get a new IP regularly.
This may put them outside of your policy.
Title: Re: Block YouTube App
Post by: sy on November 15, 2020, 08:37:09 am
Hi @Tomsauy,

Yes, please check if it matches correct policy from Reports - Connection - Live Session Explorer.
Title: Block YouTube App
Post by: Tomsauy on November 15, 2020, 09:34:34 am
Hi

I am using the free edition of Sensei so the policy apply for all devices on the network there is no rule for specific device
Title: Re: Block YouTube App
Post by: sy on November 16, 2020, 08:57:20 am
Hi @Tomsauy,

Can you share a screenshot of session detail of the android device after you used Youtube App.
Title: Re: Block YouTube App
Post by: sy on November 16, 2020, 11:49:02 pm
Hi @Tomsauy,

I can not reproduce the same issue in the lab. It seems that there is a specific situation. I got experienced that when the modem's DHCP is also opened, some services use IPv6 which leased from the modem. So please check that if this matches your issue as well. Otherwise, I would like to look into it. Please send a bug report and let's dive into it.
Title: Re: Block YouTube App
Post by: Tomsauy on November 17, 2020, 09:16:21 am
Hi @Sy

please find attached the screenshot of Sensei Live Session & nTopNG Flows both for ip 10.222.1.219

What do you mean by "modem's DHCP is also opened" ?
I have double check my DHCP config there is no IPv6 available.

Thanks for your help
Title: Re: Block YouTube App
Post by: sy on November 17, 2020, 07:37:00 pm
Hi Tomsauy,

It seems all sessions blocked. I mean when modem DHCP service is also on and it has a direct connection with client devices, Client devices can lease IPv6 from it and some services works on IPv6. Especially Google services goes on IPv6 if it is exist. If you are sure that it isn't exist in your issue, please send a bug report and I'm going to look into the logs.
Title: Re: Block YouTube App
Post by: Tomsauy on November 17, 2020, 11:06:07 pm
Thanks for those precisions, but all devices go through OPNSense router.
I have submitted a ticket, hope you can find what is wrong with my configuration.
Thanks
Title: Re: Block YouTube App
Post by: ittk on November 18, 2020, 01:38:33 pm
Thanks for those precisions, but all devices go through OPNSense router.
I have submitted a ticket, hope you can find what is wrong with my configuration.
Thanks

Good, as it's no issue from your or my side. If have all setup correctly. Result: only the webpage (access) youtube.com is getting properly blocked, but not when using the real youtube APP using e. g. Andoid smartphones. So App control seems to lack APP awareness at last for youtube... maybe many other APPs are also affected.
Title: Re: Block YouTube App
Post by: sy on November 18, 2020, 06:14:07 pm
Hi,

Unfortunately I tested with both mobile and tablet and Youtube is always blocking. In my configuration only Youtube app is selected to block. And I tried on PC with Firefox, Chrome
On Android Tablet with Youtube Application
On Android Mobile Phone with Youtube Application
On IPhone with Youtube Application.

On mobile devices Application opens but no video is playing. Sometimes only advertisements are playing and then just tries to load videos and nothing can load it.
Title: Re: Block YouTube App
Post by: ittk on November 19, 2020, 08:40:47 am
Hi,

Unfortunately I tested with both mobile and tablet and Youtube is always blocking. In my configuration only Youtube app is selected to block. And I tried on PC with Firefox, Chrome
On Android Tablet with Youtube Application
On Android Mobile Phone with Youtube Application
On IPhone with Youtube Application.

On mobile devices Application opens but no video is playing. Sometimes only advertisements are playing and then just tries to load videos and nothing can load it.

Hi,

will maybe have time tomorrow to test again and also get gback to your requested info.

But for your unterstanding i have just 1 single NIC-Port Firewall:

WAN is VLAN2 (static IPv4 only)
LAN is VLAN3 (static IPv4 only)
und OPT is untagged / no VLAN active (static Ipv4 only)

Sensei free is only active on em0_vlan3 interface and NOT on em0.

Configuration:

I have all youtube cateogries within app control on block
AND also in web control (webfilter) the pre-defined restrictive profile active.

Mabye you can reproduce the issue which given info.

And can you give me the more in depth technical insight, how you detect the youtube APP / video streams= hope not just only with just DNS-Namespace for youtube or based on just IP-Networkadresses.

Real Layer7-Detection must operate on L7 and detect the application content based information on upper layer 5 to 7 protocols.

Thanks
Title: Re: Block YouTube App
Post by: mb on November 20, 2020, 04:13:45 pm
Hi @ittk, @Tomsauy,

It seems Google has been experimenting to avoid ad-blocking mechanisms for some time now - because of ad revenue concerns.

Can you try adding Streaming/QUIC application to your list of blocked applications (from App Controls) and see if this changes anything?
Title: Re: Block YouTube App
Post by: athurdent on November 22, 2020, 08:12:14 am
Yeah, seems Youtube has been rolling out changes lately? Last week we had to turn off Ad Tracker (in App Controls) to get YouTube on Safari working on my son‘s iPad again. Interestingly, an iPhone also using Safari was working fine at the same time the iPad was not.
Now, looking at the reports, YouTube is no longer recognized properly. There is mainly Quic UDP Connecction listed now, for the iPad mostly streaming YouTube.
Title: Re: Block YouTube App
Post by: Tomsauy on November 23, 2020, 07:42:37 pm
Hi mb,

Even if I add “Quick UDP Streaming” in block app the result is the same I can continue to watch video on my android phone with YouTube app

I can see QUIC UDP flow open in ntopng and Sensei seems not be able to block it...
Title: Re: Block YouTube App
Post by: sy on November 25, 2020, 03:39:02 pm
Hi,

We are working on it and tomorrow will publish a new DB. Thanks for your understanding and patience.
Title: Re: Block YouTube App
Post by: sy on November 26, 2020, 09:07:29 pm
Hi all,

The new DB was shipped. It updates automatically every hours and you can do it manually from Status page.
Title: Re: Block YouTube App
Post by: Tomsauy on November 28, 2020, 11:14:45 pm
So bad.... the result is the same
I don’t understand why...
Title: Re: Block YouTube App
Post by: athurdent on November 29, 2020, 08:18:57 am
While I see some Youtube again now (1.19G), there are still 2.91G QUIC "unaccounted for" over the last 24h. That's from my son's iPad, and all he does ATM is watch Youtube.
QUIC hosts drill down to 74.125.104.75 (Google) and 92.224.0.0/13 (Telefonica).
Telefonica is now owned by O2, which is the DSL provider used for the iPad's traffic. So either they have setup google caching infrastructure at the ISP, or, more likely, there is some peer to peer for Youtube going on now, because the net's description is:
descr:          ADSL Pool Customers
and one of the IPs reverse resolves as:
dynamic-092-226-002-016.92.226.pool.telefonica.de
which usually is a dynamic PPPoE IP here.
Title: Re: Block YouTube App
Post by: ittk on November 29, 2020, 09:09:59 am
So bad.... the result is the same
I don’t understand why...
Can you Post all your revent Version Infos from DB ...?
Title: Re: Block YouTube App
Post by: ittk on November 29, 2020, 09:12:38 am
While I see some Youtube again now (1.19G), there are still 2.91G QUIC "unaccounted for" over the last 24h. That's from my son's iPad, and all he does ATM is watch Youtube.
QUIC hosts drill down to 74.125.104.75 (Google) and 92.224.0.0/13 (Telefonica).
Telefonica is now owned by O2, which is the DSL provider used for the iPad's traffic. So either they have setup google caching infrastructure at the ISP, or, more likely, there is some peer to peer for Youtube going on now, because the net's description is:
descr:          ADSL Pool Customers
and one of the IPs reverse resolves as:
dynamic-092-226-002-016.92.226.pool.telefonica.de
which usually is a dynamic PPPoE IP here.
Yes, did the same observation - with Vodafone as Internet Access Provider when QUIC comes Info place...
Title: Re: Block YouTube App
Post by: ittk on November 29, 2020, 09:28:11 am

QUIC hosts drill down to 74.125.104.75 (Google) and 92.224.0.0/13 (Telefonica).

This Sounds thats sensei app detection only operates on layer 3/4 detection. This is not enough for the real App detection and control used in next Generation Firewalls -  as it must operate in upper Layer 5 up to 7. You must examine up to the Layer 7 to fully catch things like Youtube Video streams
Title: Re: Block YouTube App
Post by: Tomsauy on November 30, 2020, 02:45:39 pm
So bad.... the result is the same
I don’t understand why...
Can you Post all your revent Version Infos from DB ...?

Here is a screenshot of my installed DB version
Title: Re: Block YouTube App
Post by: sy on November 30, 2020, 06:20:25 pm
Hi,

I would like to check your system. We tested different Android devices and youtube is totally blocking with App policy. Please contact via Bug report on Sensei GUI and let's check on your system.
Title: Block YouTube App
Post by: Tomsauy on November 30, 2020, 06:56:15 pm
I already have a ticket opened via report GUI, how do you want to check my system?
Title: Re: Block YouTube App
Post by: Anael on December 01, 2020, 03:57:10 pm

QUIC hosts drill down to 74.125.104.75 (Google) and 92.224.0.0/13 (Telefonica).

This Sounds thats sensei app detection only operates on layer 3/4 detection. This is not enough for the real App detection and control used in next Generation Firewalls -  as it must operate in upper Layer 5 up to 7. You must examine up to the Layer 7 to fully catch things like Youtube Video streams
In fact they always says it is 3,4 layer. We do not do ssl decryption as well. It's only SNI checking.
Title: Re: Block YouTube App
Post by: ittk on December 01, 2020, 04:53:38 pm

QUIC hosts drill down to 74.125.104.75 (Google) and 92.224.0.0/13 (Telefonica).

This Sounds thats sensei app detection only operates on layer 3/4 detection. This is not enough for the real App detection and control used in next Generation Firewalls -  as it must operate in upper Layer 5 up to 7. You must examine up to the Layer 7 to fully catch things like Youtube Video streams
In fact they always says it is 3,4 layer. We do not do ssl decryption as well. It's only SNI checking.
As you come form Fortinet you know it all. And yes, but this level (3 max layer 4) not not enough these days. So TLS up to v1.3 interception is required for the upper layer real app detection and also the more advanced malware scanning of network traffic which also totally lacks? Btw.: I don't see any real malware scan engine integration, yet now? Is it to come, soon?

And please, don't get it wrong, it's just all the good advice to further improve sensei ;)

@Tomsauy: I also had the strange issues that youtube app is not blocking while within live session view it was shown as blocked. i have fully reset of default the sensei plugin. and just enabled web control on highest level, and searched for app control (allwith youtube in it and also added quic). so far its seems to work, but currently i am not too convied it will keep so, as i have done this step already for the second time. And why cannot the APP DB really being updated with auto-update and install? Have not seen the feature, yet.
Title: Re: Block YouTube App
Post by: sy on December 04, 2020, 07:50:23 pm
Hi,

The inspection feature is in the works. Most probably early 2021.

ApplicationDB update feature is active but the latest DB config had an error and it fixed. So now Sensei updates DB automatically.
Title: Re: Block YouTube App
Post by: Tomsauy on December 04, 2020, 07:57:01 pm
Hi

After resetting all settings of Sensei, update DB and activate YouTube in app blocking, Google search and also Gmail are block......
I think there is a big trouble with last DB
Title: Re: Block YouTube App
Post by: Tomsauy on December 08, 2020, 11:19:45 pm
No update ?!
Title: Re: Block YouTube App
Post by: sy on December 09, 2020, 11:01:45 am
Hi Tomsauy,

Just released a DB. Some Google Services and Apps updated and added with 1.6.20201209014859 AppDB. Looking forward your feedback.
Title: Re: Block YouTube App
Post by: ittk on December 10, 2020, 06:29:20 am
Hi Tomsauy,

Just released a DB. Some Google Services and Apps updated and added with 1.6.20201209014859 AppDB. Looking forward your feedback.

Just updated the DB manually.

1. But why there is still no real auto-update for the DB?
2. This Opnsense Forum is completly blocked on report detected as Online Utility --> OPNSENSE, but within App Controls it is fully allowed! Web Controls are set to "High Control". Only lowering it to Moderate Control. Will workaround on it. So why does Report promtit as an APP being detected and blocked, when the issue maybe lies within the web control part and having selected the "High control" profile?
 
Its just after the full opnsense unit reboot, where this forum can be accessed for a while, but i guess when all sensei services and modules enignes are fully loaded, it will be blocked.
Title: Re: Block YouTube App
Post by: sy on December 10, 2020, 02:04:03 pm
Hi @ittk,

1- If it is enabled from the Configuration - Updates & Health, it updates automatically.
2- Web and AppDB are different. In High control, Blogs are also blocking and forum.opnsense.org category is "Blogs" in Web DB.
 
Title: Re: Block YouTube App
Post by: ittk on December 10, 2020, 04:39:29 pm
Hi @ittk,

1- If it is enabled from the Configuration - Updates & Health, it updates automatically.
2- Web and AppDB are different. In High control, Blogs are also blocking and forum.opnsense.org category is "Blogs" in Web DB.

Hi,

thanks,

1) But why is is there not in detail view which module an site was blocked, the reason for it? If its caused by web control rules (and which one) or by app controls, blocked app name rule in detail?

In Live session view it just says (classified) as blocked application Online Utility --> OPNSENSE.
Or do i miss something to get the better view which modules (web or app controls and which exact rule of it) blocked it?

2) Here you go: but it's ticked all on:

Updates and Support   full help
 Check For Updates Automatically:
Last Update Check: 11/07/2020 12:35   
 Automatically update Databases And Threat Intelligence Data:
Last Updated: 12/10/2020 06:07

Whats the update und auto-install invervall? Maybe longer ranged, so i was before it run?

This morning i have to manually check for the APP DB update und installed and reloaded it by hand, as it was not yet installed.
Title: Re: Block YouTube App
Post by: sy on December 10, 2020, 06:27:50 pm
Hi @ittk,

You can view details of the blocked session in Live Session Explorer by keeping the cursor on the blocked line, or Live Blocked Sessions Explorer.

The update time interval is in every hour. So you loaded the new version on Viewversions or with check updates?

 
Title: Re: Block YouTube App
Post by: ittk on December 10, 2020, 07:11:33 pm
Hi @ittk,

You can view details of the blocked session in Live Session Explorer by keeping the cursor on the blocked line, or Live Blocked Sessions Explorer.

The update time interval is in every hour. So you loaded the new version on Viewversions or with check updates?

With check Updates...
Title: Re: Block YouTube App
Post by: ittk on December 13, 2020, 09:02:16 am
Hi @ittk,

You can view details of the blocked session in Live Session Explorer by keeping the cursor on the blocked line, or Live Blocked Sessions Explorer.


True, but when only the blocked actions come from web controls action, the website warning page "blocked for reasons xyx" should be display to the user, as when you try to acesss adult content webpages. here when trying to access the opnsense forum, it's nothing displayed on the web browser (pahe just won't open), so it's having the same effect, like it is accessed when just the app control signatures have been applied.
Title: Re: Block YouTube App
Post by: sy on December 14, 2020, 04:24:53 pm
Hi @ittk,

Yes, The landing page works for HTTP connections yet. In early 2021 it will work for HTTPS sites as well.

Title: Re: Block YouTube App
Post by: mb on December 16, 2020, 07:36:40 pm
Adding to @sy's comment:

Along with the new landing page support, we will be providing "access pin" feature, with which you'll be able to create a PIN that will allow temporary/permanent access to a blocked connection.

We think this will be helpful in the sense that you'll be able to do whitelisting without having to visit the Sensei administration interface.

Title: Re: Block YouTube App
Post by: Anael on December 21, 2020, 10:56:01 am
that PIN option sounds amazing !