Messages

106

Web Proxy Filtering and Caching / Re: WAF app/service based whitelists

« on: September 25, 2020, 06:09:16 pm »

Thanks for patience and help!
Last question, By enabling only security rules and not choosing custom policies, location has all basic or main rules active? What's a difference between them (main and basic)?

107

Web Proxy Filtering and Caching / Re: WAF app/service based whitelists

« on: September 25, 2020, 04:55:17 pm »

Unfortunately...
Just to clarify, while in process of setting up whitelist I check errors in log and for

Code: [Select]

/logout&learning=0&vers=0.56&total_processed=1&total_blocked=1&block=1&cscore0=$policy8caca66bc2054683b0f9dcc96d4bb44c&score0=8&zone0=ARGS&id0=1206&var_name0=requesttoken,
and
/logout&learning=0&vers=0.56&total_processed=23&total_blocked=3&block=1&cscore0=$policyeeb570a227a940a7b044aac8b8faeffc&score0=16&zone0=ARGS&id0=1009&var_name0=requesttoken,I whitelist rules 1206 and 1009. Am I making it right by simply copying respective rules,  changing name and description, setting them to whitelist and removing any arguments and values?
Basic or Main rule? For example as it is in att. screenshot.


One more thing, I can't identify correctly rule in following string:

Code: [Select]

&learning=0&vers=0.56&total_processed=62&total_blocked=1&block=1&zone0=BODY&id0=11&var_name0=,Is it Body 11? Doesn't seem to be correct ID...

108

Web Proxy Filtering and Caching / Re: WAF app/service based whitelists

« on: September 25, 2020, 03:25:03 pm »

I've found this: https://ownyourbits.com/2017/03/23/modsecurity-web-application-firewall-for-nextcloud/
Is there any way to translate Modsecurity rules IDs to NAXSi IDs?

Code: [Select]

<Directory /var/www/nextcloud/>
# VIDEOS
  SecRuleRemoveById 958291             # Range Header Checks
  SecRuleRemoveById 981203             # Correlated Attack Attempt


  # PDF
  SecRuleRemoveById 950109             # Check URL encodings


  # ADMIN (webdav)
  SecRuleRemoveById 960024             # Repeatative Non-Word Chars (heuristic)
  SecRuleRemoveById 981173             # SQL Injection Character Anomaly Usage
  SecRuleRemoveById 981204             # Correlated Attack Attempt
  SecRuleRemoveById 981243             # PHPIDS - Converted SQLI Filters
  SecRuleRemoveById 981245             # PHPIDS - Converted SQLI Filters
  SecRuleRemoveById 981246             # PHPIDS - Converted SQLI Filters
  SecRuleRemoveById 981318             # String Termination/Statement Ending Injection Testing
  SecRuleRemoveById 973332             # XSS Filters from IE
  SecRuleRemoveById 973338             # XSS Filters - Category 3
  SecRuleRemoveById 981143             # CSRF Protections ( TODO edit LocationMatch filter )


  # COMING BACK FROM OLD SESSION
  SecRuleRemoveById 970903             # Microsoft Office document properties leakage


  # NOTES APP
  SecRuleRemoveById 981401             # Content-Type Response Header is Missing and X-Content-Type-Options is either missing or not set to 'nosniff'
  SecRuleRemoveById 200002             # Failed to parse request body


  # UPLOADS ( 5 MB max excluding file size )
  SecRequestBodyNoFilesLimit 5242880


  # GENERAL
  SecRuleRemoveById 960017             # Host header is a numeric IP address


  # REGISTERED WARNINGS, BUT DID NOT HAVE TO DISABLE THEM
  #SecRuleRemoveById 981220 900046 981407
  #SecRuleRemoveById 981222 981405 981185 981184
</Directory>

109

Web Proxy Filtering and Caching / WAF app/service based whitelists

« on: September 25, 2020, 01:34:11 am »

Do you know where to find application based Nginx WAF whitelists? I mean tailor made set of rules for i.e. Sogo, Nextcloud and so on...
Thanks!

110

Web Proxy Filtering and Caching / Re: HOWTO insert custom headers - Nginx

« on: September 24, 2020, 11:57:06 pm »

Thanks @Fright!
It works well with proxy_redirect and independently from WAN port number. And it is such a simple solution...
My goal is to set services in OPNsense mainly via webgui for ease of future maintenance. I used to have USG 4 Pro from Ubiquiti but that gateway's  webgui functions were lacking seriously  - most of a bit more advanced functions, had to be configured via json file (including NAT, OpenVPN and so on). 


One more thing, is it possible to use WAF whitelist? I mean If I set it to allow matching requests, all other requests will be rejected?

111

Zenarmor (Sensei) / Re: Sensei on OPNsense - Application based filtering

« on: September 24, 2020, 04:31:48 am »

This is being done for Mongodb. Elasticsearch will be next for 1.7. Thanks for the suggestion.

Thanks @mb.
Thus which DB is recommended?

112

Zenarmor (Sensei) / Re: Sensei on OPNsense - Application based filtering

« on: September 24, 2020, 02:38:31 am »

A good discussion on why you might consider offloading Elasticsearch reporting:
https://forum.opnsense.org/index.php?topic=19266.msg88593#msg88593

In some scenarios it makes sense to offload DB, but also would be very convenient if Sensei runs periodical checks of DB and if required, some basic auto repair plus reports any inconsistency... 

113

Web Proxy Filtering and Caching / Re: HOWTO insert custom headers - Nginx

« on: September 24, 2020, 02:06:31 am »

ok. it remains to understand which port the backend is listening to. and is there a url changing between user request and request from nginx to backend.
if the problem is only in the absence of a port in the redirect location then you can try

Code: [Select]

proxy_redirect https://server.domain.com/ https://server.domain.com:4443/;


Backend listens on port 80.
Above proxy_redirect works in location block, but how to make "if" statement to kick in redirect only when client makes request on port 4443 which in turn is transparent for Nginx since firewall forwards such request to Nginx... So, as you mentioned it must be based on redirect code 307 or 302? Otherwise LAN clients can't login using regular https port...
Below is curl request/response, it's 302 redirect and it's the same regardless request origin (LAN/WAN):

* Connection state changed (MAX_CONCURRENT_STREAMS == 128)!
< HTTP/2 302 
< server: nginx
< date: Wed, 23 Sep 2020 23:59:15 GMT
< content-type: text/html; charset=UTF-8
< content-length: 0
< cache-control: no-store, no-cache, must-revalidate
< content-security-policy: default-src 'self'; script-src 'self' style-src 'self' 'unsafe-inline'; frame-src *; img-src * data: blob:; font-src 'self' data:; media-src *; connect-src *; object-src 'none'; base-uri 'self';
< location: https://server.domain.com/login

114

Zenarmor (Sensei) / Re: Sensei on OPNsense - Application based filtering

« on: September 22, 2020, 11:25:04 pm »

It looks like, OPNsense will land them on 20.7.4 at the earliest. I've heard from @franco that there'll be another netmap test kernel based on 20.7.3. So, not for 20.7.3 for sure.

Thus it means all users with vmx interfaces must wait at least until 20.7.4 is released?

Both Mongo and Elastic do lots of buffered I/O for performance reasons. In case of an abrupt shutdown, they have no way of recovering in-memory data which is not yet written to disk.
Sensei -> Configuration -> Reporting & Data -> Reset Reporting will try to recover broken indexes, if not, they'll reset broken indexes.

Since I did complete reinstallation, I don't need to reset DB once again?

115

Web Proxy Filtering and Caching / Re: HOWTO insert custom headers - Nginx

« on: September 22, 2020, 11:13:44 pm »

why? it should work imho. if after users auth backend server send 307 with location: https://server.domain.com/welcome.html,  proxy_redirect  will replace loaction with https://server.domain.com:4443/welcome.html for you. then client hit the 4443 port

Ok, I'm not good at all in proxy stuff :-)
Nginx listens on 443, firewall opens WAN port 4443 and forwards traffic to Nginx port 443.
So, if it's possible, can I somehow configure such redirect in webgui or I must go for hook file? Could you show example for above case?

116

Web Proxy Filtering and Caching / Re: HOWTO insert custom headers - Nginx

« on: September 22, 2020, 08:47:25 pm »

look at backend logs for responses after user authentication. i think that authentication form in /login page redirects user to absolute url after authentication. if so you can try proxy_redirect directive in location block (http://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_redirect) (https://stackoverflow.com/questions/20254456/intercepting-backend-301-302-redirects-proxy-pass-and-rewriting-to-another-loc)

In general, I'm confused here. Since port forwarding is done by firewall (fw 4443<->443 Nginx), thus for Nginx it's transparent. But looking at curl reply, in header there's:
location: https://server.domain.com/login


I've tried in URL rewriting to set
Source: $scheme://server.domain.com:4443/(.*)$
Destination: $scheme://$http_host/$1
Flag: Redirect
and activate it in location but that doesn't change a bit.... Does $http_host contain domain and port?


EDIT:

Since firewall/NAT forwarding is transparent for Nginx, Nginx is not aware of port, client is communicating on, so in my humble opinion there's no way to change header in location response, neither using redirect nor proxy_redirect.... Unfortunately

117

Web Proxy Filtering and Caching / Re: HOWTO insert custom headers - Nginx

« on: September 22, 2020, 04:14:00 am »

I can't solve small issue: If I need an external access, I'm used to expose services running inside LAN on router's WAN interface but using random port numbers. Such port forwarding works fine but either upstream or Nginx keep rewriting random port to standard 443 and communication is lost. I mean when I have set forwarding i.e. port 4443 to 443: page https://server.domain.com:4443 gets rewritten after typing in logon credentials to https://server.domain.com/login and of course lost comms error is displayed. When I manually amend port to 4443 and and hit enter page is opened and user is logged in. Will "port_in_redirect off" in server block be permanent a solution? Any cons?

118

Zenarmor (Sensei) / Re: Sensei on OPNsense - Application based filtering

« on: September 21, 2020, 10:22:50 pm »

20.7.2-netmap kernel looks fine. I've just seen your correspondance with our support team. I guess you'll be waiting for the release kernel

Exactly  , I would have tried out test kernel but in such a case I need physical access to router. Will all changes be included in 20.7.3 or rather later updates?
And for now I removed completely Sensei and have it reinstalled. Maybe DB had been corrupted during unsuccessful update to 20.7 and following VM's snapshot restoration?

119

Zenarmor (Sensei) / Re: Sensei on OPNsense - Application based filtering

« on: September 21, 2020, 03:48:07 am »

2020-09-09T04:31:07   kernel: 667.875025 [1180] netmap_grab_packets bad pkt at 390 len 0
2020-09-09T04:31:07   kernel: 667.875016 [1180] netmap_grab_packets bad pkt at 389 len 0
2020-09-09T04:31:07   kernel: 667.875008 [1180] netmap_grab_packets bad pkt at 388 len 0
2020-09-09T04:31:07   kernel: 667.875001 [1180] netmap_grab_packets bad pkt at 387 len 0
2020-09-09T04:31:07   kernel: 667.874992 [1180] netmap_grab_packets bad pkt at 386 len 0
2020-09-09T04:31:07   kernel: 667.874306 [ 277] vmxnet3_netmap_rxsync 130 skipped! idx 46
2020-09-09T04:31:07   kernel: vmx1: watchdog timeout on queue 0
2020-09-09T04:31:02   eastpect[8308]: nm1::vmx1^: permanently promiscuous mode enabled
2020-09-09T04:31:02   eastpect[8308]: nm0::vmx1: permanently promiscuous mode enabled
What surprising me is that all has been working fine for months, I had done no changes in setup, no new packages were installed and all of sudden this problem appears. I know it's net map but could it be triggered somehow by Sensei which inspects parent interface vmx1?
Shall I reinstall Sensei, would it help?

@mb just to let you know that above issue must be caused or triggered by Sensei. I can reinstate LAN communication by simply stopping Sensei Packet Engine (fyi OPNsense is still on 20.1.9 and Sensei 1.5.2).
I'm writing this to ask whether this issue has been addressed in new release?

120

Web Proxy Filtering and Caching / Re: HOWTO insert custom headers - Nginx

« on: September 18, 2020, 03:46:52 pm »

Anyway, are there any size limits for error logs?

you can edit template and add size limit (in kilobytes)
but I really think it's not about the newsyslog settings. you need to look at the log and figure out what is the reason for so many errors.

Does it look sensible and correct (10000k)?

Code: [Select]

root@OPNsense:/usr/local/etc/nginx # cat /usr/local/opnsense/service/templates/OPNsense/Nginx/newsyslog.conf
# logfilename                   [owner:group]   mode    count size      when    flags   [/pid_file]               [sig_num]
{% if helpers.exists('OPNsense.Nginx') %}
/var/log/nginx/*access.log   www:www     640     14       10000      @T00     GZB      /var/run/nginx.pid       30
/var/log/nginx/*error.log   www:www     640     14       10000      @T00     GZB      /var/run/nginx.pid       30
{% endif %}
 And you are right, I should investigate it further, but like I said, my main task was not to loose connection :-)
Anyway, currently error log for that particular upstream/server stays empty; maybe it was just one off glitch...