[solved] nginx auth issues with Exchange 2016/IIS 401 loop

[Fright]

imho any option should work.
The second option seems to me the most correct and nifty. but you will need to edit two templates, create two dirs and add two files, respectively. plus templates can be overwritten during updates and this will need to be monitored (it would be nice to add hooks to templates so that we don't have to do this. but not yet)
i will try to write steps when you choose one )

[klamath]

Thanks for the quick follow up, lets shoot for doing it the correct way and go with choice 2.  I pay for the business edition of opnsense (I like to donate to projects I support) so upgrades shouldn't be too bad to deal with.

[Fright]

OK. then let's try
1. You need to create a hooks in the http block to read additional configuration (maps in this case) and in location block to read additional config (headers directives in this case):
in http template file

Code: [Select]

/usr/local/opnsense/service/templates/OPNsense/Nginx/http.conf
right above the line

Code: [Select]

# TODO add when core is ready for allowing nginx to serve the web interfaceadd a hook line. so it should look like

Code: [Select]

include http_post/*.conf;
# TODO add when core is ready for allowing nginx to serve the web interface
in location template file

/usr/local/opnsense/service/templates/OPNsense/Nginx/location.conf

right above the last curly brace add a hook line for location _post-config.
it should look like

Code: [Select]

{% endif %}{# honeypot #}
    include {{ location['@uuid'] }}_post/*.conf;
}

then click Apply in Nginx->Configuration->General Settings in GUI.
the templates should apply and in the config file

Code: [Select]

/usr/local/etc/nginx/nginx.conf
you should see the hooks that appear:
"include http_post/*.conf;" above upstreams part and
"include *someUIDdigits*_pre/*.conf;" in the end of each location
if everything worked out, then you can create dirs and files

2. Add extra config
In /usr/local/etc/nginx/ dir make "http_post" dir and "yourlocationUID_post" dir.
the last dir name you can take from hook string in your location block in nginx.conf.
in "http_post" dir you can put a file with the name say maps.conf with content:

Code: [Select]

map $status $forceBasic{
        401    'Basic realm=mail.xxx.com';
}in ""yourlocationUID_post"" dir you can put a file with the name say autodisco_location.conf with content:

Code: [Select]

proxy_hide_header WWW-Authenticate;
add_header WWW-Authenticate $forceBasic always;
hit Apply in Nginx->Configuration->General Settings in GUI.
if everything worked out, then the M$ connectivity test should be happy )

[klamath]

Perfect sir, and many thanks, I will try this week and get back to you!

OK. then let's try
1. You need to create a hooks in the http block to read additional configuration (maps in this case) and in location block to read additional config (headers directives in this case):
in http template file

Code: [Select]

/usr/local/opnsense/service/templates/OPNsense/Nginx/http.conf
right above the line

Code: [Select]

# TODO add when core is ready for allowing nginx to serve the web interfaceadd a hook line. so it should look like

Code: [Select]

include http_post/*.conf;
# TODO add when core is ready for allowing nginx to serve the web interface
in location template file

/usr/local/opnsense/service/templates/OPNsense/Nginx/location.conf

right above the last curly brace add a hook line for location _post-config.
it should look like

Code: [Select]

{% endif %}{# honeypot #}
    include {{ location['@uuid'] }}_post/*.conf;
}

then click Apply in Nginx->Configuration->General Settings in GUI.
the templates should apply and in the config file

Code: [Select]

/usr/local/etc/nginx/nginx.conf
you should see the hooks that appear:
"include http_post/*.conf;" above upstreams part and
"include *someUIDdigits*_pre/*.conf;" in the end of each location
if everything worked out, then you can create dirs and files

2. Add extra config
In /usr/local/etc/nginx/ dir make "http_post" dir and "yourlocationUID_post" dir.
the last dir name you can take from hook string in your location block in nginx.conf.
in "http_post" dir you can put a file with the name say maps.conf with content:

Code: [Select]

map $status $forceBasic{
        401    'Basic realm=mail.xxx.com';
}in ""yourlocationUID_post"" dir you can put a file with the name say autodisco_location.conf with content:

Code: [Select]

proxy_hide_header WWW-Authenticate;
add_header WWW-Authenticate $forceBasic always;
hit Apply in Nginx->Configuration->General Settings in GUI.
if everything worked out, then the M$ connectivity test should be happy )

[klamath]

Howdy!

I got some time to try this, hitting a snag on the location config file not generating:

/usr/local/opnsense/service/templates/OPNsense/Nginx/location.conf

Code: [Select]

{% endif %}{# honeypot #}
    include {{ location['@uuid'] }}_post/*.conf;
}


root@cerberus:~ # grep include /usr/local/etc/nginx/nginx.conf
include       mime.types;
js_include /usr/local/opnsense/scripts/nginx/ngx_functions.js;
include http_post/*.conf;
# include nginx_web.conf;
include opnsense_http_vhost_plugins/*.conf;
    include opnsense_stream_vhost_plugins/*.conf;

[Fright]

Hi
did you add "    include {{ location['@uuid'] }}_post/*.conf;" to "/usr/local/opnsense/service/templates/OPNsense/Nginx/location.conf"  and "Apply" after that?
you need to add it right above the last curly brace (line #214)

[klamath]

Yes I did, nginx tries to start when I hit apply:

Code: [Select]

root@cerberus:~ # grep -n -A1 -B1 _post /usr/local/opnsense/service/templates/OPNsense/Nginx/location.conf
208-{% endif %}{# honeypot #}
209:    include {{ location['@uuid'] }}_post/*.conf;
210-}

I do see some warnings appear with nginx starts but it seems that is around SSL cert joining?

[09-Feb-2021 09:49:16 America/Chicago] PHP Warning:  Invalid argument supplied for foreach() in /usr/local/opnsense/scripts/nginx/setup.php on line 184

[klamath]

I just blew away the installed package and reinstalled, same issue still, this is the last few lines of the running config.

root@cerberus:~ # tail -n5 /usr/local/etc/nginx/nginx.conf
    include opnsense_stream_vhost_plugins/*.conf;

}
# mail {
# }

[Fright]

the last few lines of the running config
include opnsense_stream_vhost_plugins/*.conf;

the rest of the config is generated only if nginx is enabled
(if OPNsense.Nginx.general.enabled is defined and OPNsense.Nginx.general.enabled == '1')

is nginx itself enabled?)

[klamath]

Perfect, I was trying to stage all the changes before applying the config.  For matching up the UUID for the directory, what should I look for as I have other services besides exchange:

Code: [Select]

root@cerberus:~ # grep include /usr/local/etc/nginx/nginx.conf
include       mime.types;
js_include /usr/local/opnsense/scripts/nginx/ngx_functions.js;
include http_post/*.conf;
# include nginx_web.conf;
include opnsense_http_vhost_plugins/*.conf;
    #include tls.conf;
      include       fastcgi_params;
      include        fastcgi_params;
    include b21a09c6-db5d-4ce0-bfe0-dd7e31d89811_pre/*.conf;
    include 7b884db8-eafa-43d0-bdae-ec4a66a97cad_post/*.conf;
    include b21a09c6-db5d-4ce0-bfe0-dd7e31d89811_post/*.conf;
    #include tls.conf;
      include       fastcgi_params;
      include        fastcgi_params;
    include 7b844599-dbd4-4b45-ad56-e22dd094d6d5_pre/*.conf;
    include 016848f7-b0de-401b-b961-b7bfeed575ab_post/*.conf;
    include 7b844599-dbd4-4b45-ad56-e22dd094d6d5_post/*.conf;
    #include tls.conf;
      include       fastcgi_params;
      include        fastcgi_params;
    include f94e4a2e-e9ea-419b-a5b8-763890eaa89b_pre/*.conf;
    include d9acf7d4-f0b5-4530-b574-ad4b28375e18_post/*.conf;
    include f94e4a2e-e9ea-419b-a5b8-763890eaa89b_post/*.conf;
    #include tls.conf;
      include       fastcgi_params;
      include        fastcgi_params;
    include f857a060-bf3e-4d6d-af2a-5073ee117b2d_pre/*.conf;
    include cc8de2e8-3e31-4994-bd0c-5712a874fb04_post/*.conf;
    include f857a060-bf3e-4d6d-af2a-5073ee117b2d_post/*.conf;
    #include tls.conf;
      include       fastcgi_params;
      include        fastcgi_params;
    include 040f7fe9-396e-4b8e-8e4c-19a3eff357c4_pre/*.conf;
    include 5eeb9519-02e7-4dee-9368-f5dd50f5779d_post/*.conf;
    include 040f7fe9-396e-4b8e-8e4c-19a3eff357c4_post/*.conf;
    #include tls.conf;
      include       fastcgi_params;
      include        fastcgi_params;
    include fcdd1729-8503-4055-80e3-cf74112ca928_pre/*.conf;
    include 8498a998-7bbf-4401-80aa-7498170d3a34_post/*.conf;
    include fcdd1729-8503-4055-80e3-cf74112ca928_post/*.conf;
    include opnsense_stream_vhost_plugins/*.conf;



[Fright]

what should I look for as I have other services besides exchange

in nginx.conf in the end of "autodiscovery" location block you can see the hook for this particular location with location's UUID in folder name

[klamath]

Perfect,

I got to this part of the remote test and it is still failing:

Code: [Select]

Attempting to ping the MAPI Mail Store endpoint with identity: xxxxx-9188-4474-9811-0ef5db77cf19@xxx.com:6001.
The attempt to ping the endpoint failed.
Additional Details
An RPC error was thrown by the RPC Runtime process. Error 1818 CallCancelled
RPC Status: 1818 CallCancelled
Timestamp: 2/9/2021 5:33:20 PM
Generating Component: 14 (WinHttp)
Status: 1818
Detection Location: 1390 (HTTP2ClientVirtualConnection__ClientOpenInternal10)
Flags: 0
Parameters:
30000



[klamath]

I got it working!  I had to turn off response and request buffering, and set the Maximum Body Size to 2G.

[klamath]

Do you have any recommended rules that should be disabled in WAF?  It seems that just enabling WAF without any rules selected makes the remote connectivity tester to fail.  I don't see rules blocking in waf_denied.access.log

Code: [Select]

168.61.212.41 - - [09/Feb/2021:12:22:14 -0600] "POST /Autodiscover/Autodiscover.xml HTTP/1.1" 405 150 "-" "Microsoft Office/15.0 (Windows NT 6.2; Microsoft Outlook 15.0.4615; Pro; MS Connectivity Analyzer)" "-"
168.61.212.41 - - [09/Feb/2021:12:22:14 -0600] "POST /Autodiscover/Autodiscover.xml HTTP/1.1" 405 175 "-" "Microsoft Office/15.0 (Windows NT 6.2; Microsoft Outlook 15.0.4615; Pro; MS Connectivity Analyzer)" "-"
168.61.212.41 - - [09/Feb/2021:12:22:15 -0600] "POST /Autodiscover/Autodiscover.xml HTTP/1.1" 405 150 "-" "Microsoft Office/15.0 (Windows NT 6.2; Microsoft Outlook 15.0.4615; Pro; MS Connectivity Analyzer)" "-"
168.61.212.41 - - [09/Feb/2021:12:22:15 -0600] "POST /Autodiscover/Autodiscover.xml HTTP/1.1" 405 175 "-" "Microsoft Office/15.0 (Windows NT 6.2; Microsoft Outlook 15.0.4615; Pro; MS Connectivity Analyzer)" "-"


[Fright]

request buffering, and set the Maximum Body Size to 2G

not sure if this is relevant but glad it works )

enabling WAF without any rules selected makes the remote connectivity tester to fail

HTTP/1.1" 405 150 - 405 is a "Method not allowed"
doesn't look like naxsi work. this is something else. need to see the analyzer log

for naxsi it is highly desirable to start working with Leraning Mode enabled and analyze the logs to whitelist some rules (if I remember correctly I turned off the rules 16,17, 11 and some others. but may be it was for sharepoint))