Sensei on OPNsense - Application based filtering

[sol]

Thank you!

[opnsenseuser]

rene, i was able to reproduce the issue. thanks for the hand. 1.1_4 coming up shortly.

that is fast. what´s the problem?

[mb]

In regards of dns: is it maybe dnscrypt proxy which interfers here?

sol, the issue with rene is different. yes, if you have dns encryption most probably this is the reason.

[mb]

that is fast. what´s the problem?

rene, it was a sanity check going wrong because of a missing if condition ;) Fix was easy, 1.1_4 hotfix is out. Enjoy.

sol, we're thinking of implementing "lazy dns resolution" for these cases like dns encryption. This will allow Sensei to do realtime dns query for any ip addresses for which it does not have a dns mapping in its cache. Most probably it'll ship Q2 2020.

[opnsenseuser]

that is fast. what´s the problem?

rene, it was a sanity check going wrong because of a missing if condition ;) Fix was easy, 1.1_4 hotfix is out. Enjoy.

sol, we're thinking of implementing "lazy dns resolution" for these cases like dns encryption. This will allow Sensei to do realtime dns query for any ip addresses for which it does not have a dns mapping in its cache. Most probably it'll ship Q2 2020.

you are the best. thx for your really fast response.i´ll test this later!. :-)

[opnsenseuser]

that is fast. what´s the problem?

rene, it was a sanity check going wrong because of a missing if condition ;) Fix was easy, 1.1_4 hotfix is out. Enjoy.

works. thx very much!! :-)

2 more questions:

1. is there a way to make a custom block html template? and perhaps upload it?
2. i get this error message in System: Firmware: Reporter

Code Select Expand

[20-Nov-2019 15:50:33 Europe/Vienna] PHP Warning:  explode() expects parameter 2 to be string, array given in /usr/local/opnsense/mvc/app/controllers/OPNsense/Sensei/Api/WidgetController.php on line 175
[20-Nov-2019 15:50:33 Europe/Vienna] PHP Warning:  Invalid argument supplied for foreach() in /usr/local/opnsense/mvc/app/controllers/OPNsense/Sensei/Api/WidgetController.php on line 176
[20-Nov-2019 15:50:33 Europe/Vienna] PHP Warning:  explode() expects parameter 2 to be string, array given in /usr/local/opnsense/mvc/app/controllers/OPNsense/Sensei/Api/WidgetController.php on line 181
[20-Nov-2019 15:50:33 Europe/Vienna] PHP Warning:  Invalid argument supplied for foreach() in /usr/local/opnsense/mvc/app/controllers/OPNsense/Sensei/Api/WidgetController.php on line 182

[mb]

Hi rene, you're all welcome. custom landing page is available within Premium Features. SOHO Edition is coming this week.

Dashboard widget error got already fixed in 1.2, which will also ship this week :)

[opnsenseuser]

Hi rene, you're all welcome. custom landing page is available within Premium Features. SOHO Edition is coming this week.

Dashboard widget error got already fixed in 1.2, which will also ship this week :)

is there no standard block template in the free edition ?. because the message that I get when blocking a page is a connection error page. It is therefore difficult to determine if this is a real connection error or not.
the html block template that I found did not work. or is it intended?

best regards, rene

supplement:
I noticed now, if I use "app controls" and block for example, facebook, then there is no html block template but only a connection error page (see my screenshot). if I block a page under "web control", then comes the block template. Is it wanted like that? best regards, rene

[tusc]

So I'm still experiencing issue where traffic completely halts shortly after the engine service is started. I never could figure out the problem so didn't use this for a while. I'm now on the latest version and it's still happening. I have a 4 port intel card where igb0 is LAN and igb1 is WAN. There's an onboard Realtek I'm not using (re0).

Searching in /usr/local/sensei/log/active I see this in the logs

Code Select Expand


root@OPNsense:/usr/local/sensei/log/active # egrep igb main*
main_20191119T000000.log:2019-11-19T10:45:28 INFO: Worker [@0,Bridged,Lan=netmap@igb0,Wan=netmap@igb0^,Queue=0,Cpu=1
main_20191119T000000.log:2019-11-19T21:18:49 INFO: Worker [@0,Bridged,Lan=netmap@igb0,Wan=netmap@igb0^,Queue=0,Cpu=1
main_20191120T000000.log:2019-11-20T19:16:42 INFO: Worker [@0,Bridged,Lan=netmap@igb0,Wan=netmap@igb0^,Queue=0,Cpu=1


Why is WAN referencing igb0^? Shouldn't it be igb1?

If I grep for igb1 in the directory nothing comes back.

Here's another output from a worker logfile:

Code Select Expand


root@OPNsense:/usr/local/sensei/log/active # egrep igb worker0_20191120T000000.log | tail
2019-11-21T14:57:19 INFO:               Stats LAN igb0:0 [ 4024 pkts, 0 drp, 610.74 KB]
2019-11-21T14:57:19 INFO:               Stats LAN igb0:1 [ 109564 pkts, 0 drp, 150.78 MB]
2019-11-21T14:57:19 INFO:               Stats LAN igb0:2 [ 27493 pkts, 0 drp, 2.00 MB]
2019-11-21T14:57:19 INFO:               Stats LAN igb0:3 [ 33916 pkts, 0 drp, 2.54 MB]
2019-11-21T14:57:19 INFO:               Stats WAN igb0^ [ 239646 pkts, 0 drp, 177.91 MB]
2019-11-21T14:57:20 INFO:               Stats LAN igb0:0 [ 4024 pkts, 0 drp, 610.74 KB]
2019-11-21T14:57:20 INFO:               Stats LAN igb0:1 [ 109564 pkts, 0 drp, 150.78 MB]
2019-11-21T14:57:20 INFO:               Stats LAN igb0:2 [ 27493 pkts, 0 drp, 2.00 MB]
2019-11-21T14:57:20 INFO:               Stats LAN igb0:3 [ 33917 pkts, 0 drp, 2.54 MB]
2019-11-21T14:57:20 INFO:               Stats WAN igb0^ [ 239646 pkts, 0 drp, 177.91 MB]


Let me know what else I can provide to help troubleshoot this as I've noticed others have posted a similar problem. Thanks.

[mb]

Hi Rene,

Yes, customizable block page is available in Premium.

1. With regard to how we display block page: we display Block Page only if it is an HTTP connection.
2. For HTTPS connections, since TLS comes early and client and server does not yet speak HTTP, we cannot display.
3. For Application control, we do not display since it might be a connection which does not speak HTTP.

For the third item, I think there is a window of improvement there; since we can still detect if it is HTTP
and therefore we can display a block page.

For HTTPS connections, block pages will be available along with TLS feature.

[mb]

Hi @tusc,

WAN in that file is an internal Sensei terminology and it is different from general firewall terminology. Sensei acts like a bridge connecting hardware rings of the ethernet driver and the Operating System network stack (with the help of netmap). Taking into account the fact that we're protecting LAN-facing interfaces, Sensei considers the Operating System side of the "virtual bridge" as WAN since packets going to/coming from that way is Internet-bound.

It is expected that packet flow can pause a 2-5 seconds during engine restarts. This is because once sensei starts running it initializes the interfaces in netmap mode which -in turn- causes them to go down/up.

If it halts the packet flow permanently, this is very interesing, which I would definitely want to have a look. Can you PM me so that we dive into this?

[donatom3]

@MB

How does soho work with the 15 device limit for those of us with well over that on our home networks?
Do we pick and choose what's protected or is it any device that's on the protected interface?

[robvanhooren]

hi, fresh install, and I'm getting a ton of 'index not found exception' errors, with a lot of sensei panels displaying a red error box.

"An error occurred while report is being loaded!"

details and log excerpt below.

thoughts?

thanks.

Code Select Expand

{
  "error": {
    "root_cause": [
      {
        "type": "index_not_found_exception",
        "reason": "no such index",
        "resource.type": "index_or_alias",
        "resource.id": "alert_all",
        "index_uuid": "_na_",
        "index": "alert_all"
      }
    ],
    "type": "index_not_found_exception",
    "reason": "no such index",
    "resource.type": "index_or_alias",
    "resource.id": "alert_all",
    "index_uuid": "_na_",
    "index": "alert_all"
  },
  "status": 404
}



-----8<-----{snip}-----8<-----
/usr/local/sensei/log/active

ipdr_streamer.log:2019-11-22T00:43:47.637231 response: {"took":0,"errors":true,"items":[{"index":{"_index":"http_write","_type":"http","_id":null,"status":404,"error":{"type":"index_not_found_exception","reason":"no such index","resource.type":"index_expression","resource.id":"http_write","index_uuid":"_na_","index":"http_write"}}},{"index":{"_index":"http_write","_type":"http","_id":null,"status":404,"error":{"type":"index_not_found_exception","reason":"no such index","resource.type":"index_expression","resource.id":"http_write","index_uuid":"_na_","index":"http_write"}}},{"index":{"_index":"http_write","_type":"http","_id":null,"status":404,"error":{"type":"index_not_found_exception","reason":"no such index","resource.type":"index_expression","resource.id":"http_write","index_uuid":"_na_","index":"http_write"}}}]}

[Quetschwalze]

Love the plugin!
Will there be a monthly option for paid home use?

[opnsenseuser]

Hi Rene,

Yes, customizable block page is available in Premium.

1. With regard to how we display block page: we display Block Page only if it is an HTTP connection.
2. For HTTPS connections, since TLS comes early and client and server does not yet speak HTTP, we cannot display.
3. For Application control, we do not display since it might be a connection which does not speak HTTP.

For the third item, I think there is a window of improvement there; since we can still detect if it is HTTP
and therefore we can display a block page.

For HTTPS connections, block pages will be available along with TLS feature.

thx for your information. this plugin is really really great!. great work! :-)