Hi,
I'm fighting for several days now to get haproxy (as a reverse proxy) running on my opnsense firewall with https traffic.
The actual setup is the following:
WAN (with static IP)
---> OPNSense / HA reverse Proxy (on virtual IP)
-----> Webserver for domain1
-----> Webserver for domain2
The basic setup with haproxy is working pretty good with unencrypted http traffic, but for https I can't get the rules working.
The SSL traffic should be passed directly through to the Webservers wich handels the encryption by thereself.
I have configured the backend pools / rules / conditions and frontends divided in ssl and non ssl traffic.
If I set e.g. the condition for server 1 as negotiated (then every incomming request will be forwarded to this backend) I can connect to it over https without a problem, therefore I assumed there must be some setting in my haproxy configuration that prevent the backend choise according to the requested SNI uri.
My config:
#
# Automatically generated configuration.
# Do not edit this file manually.
global
# NOTE: Could be a security issue, but required for some feature.
uid 80
gid 80
chroot /var/haproxy
daemon
stats socket /var/run/haproxy.socket level admin
nbproc 1
tune.ssl.default-dh-param 1024
spread-checks 0
tune.chksize 16384
tune.bufsize 16384
tune.lua.maxmem 0
log /var/run/log local0 info
ssl-default-bind-options no-sslv3 no-tlsv10 no-tls-tickets
ssl-default-bind-ciphers ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-CHACHA20-POLY1305:ECDHE-RSA-CHACHA20-POLY1305:ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES256-SHA384:ECDHE-RSA-AES256-SHA384:ECDHE-ECDSA-AES128-SHA256:ECDHE-RSA-AES128-SHA256
defaults
log global
option redispatch -1
timeout client 30s
timeout connect 30s
timeout server 30s
retries 3
# Frontend: PROXY_DISPATCHER (http domain dispatcher)
frontend PROXY_DISPATCHER
bind 192.168.10.10:80 name 192.168.10.10:80
mode http
option http-keep-alive
# tuning options
timeout client 30s
# logging options
# ACL: remote_xyz
acl acl_5ea7241e265c45.35734629 hdr_sub(host) -i remote.xyz.com
# ACTION: RULE_xyz
use_backend xyz_Server_POOL if acl_5ea7241e265c45.35734629
# ACL: zz_xyz_de
acl acl_5ea724a3355897.03132566 hdr_sub(host) -i zz.xyz.de
# ACTION: RULE_zz_Public
use_backend zz_Public_Webserver_POOL if acl_5ea724a3355897.03132566
# Frontend: PROXY_DISPATCHER_SSL (https domain dispatcher)
frontend PROXY_DISPATCHER_SSL
bind 192.168.10.10:443 name 192.168.10.10:443
mode tcp
# tuning options
timeout client 30s
# logging options
# ACL: zz_xyz_de_SSL
acl acl_5eab1d24347657.98217236 req.ssl_sni -m sub -i zz.xyz.de
# ACTION: RULE_zz_Public_SSL
use_backend zz_Public_Webserver_POOL_SSL if acl_5eab1d24347657.98217236
# ACL: remote_xyz_SSL
acl acl_5eab1d00637479.15142847 req.ssl_sni -m sub -i remote.xyz.com
# ACTION: RULE_xyz_SSL
use_backend xyz_Server_POOL_SSL if acl_5eab1d00637479.15142847
# WARNING: pass through options below this line
tcp-request inspect-delay 10s
# Backend: xyz_Server_POOL ()
backend xyz_Server_POOL
# health checking is DISABLED
mode http
balance source
# stickiness
stick-table type ip size 50k expire 30m
stick on src
# tuning options
timeout connect 30s
timeout server 30s
server xyz_Server 192.168.112.105:80
# Backend: zz_Public_Webserver_POOL ()
backend zz_Public_Webserver_POOL
# health checking is DISABLED
mode http
balance source
# stickiness
stick-table type ip size 50k expire 30m
stick on src
# tuning options
timeout connect 30s
timeout server 30s
server zz_Ubuntu01 192.168.112.111:80
# Backend: xyz_Server_POOL_SSL ()
backend xyz_Server_POOL_SSL
# health checking is DISABLED
mode tcp
balance source
# stickiness
stick-table type ip size 50k expire 30m
stick on src
# tuning options
timeout connect 30s
timeout server 30s
server xyz_Server_SSL 192.168.112.105:443
# Backend: zz_Public_Webserver_POOL_SSL ()
backend zz_Public_Webserver_POOL_SSL
# health checking is DISABLED
mode tcp
balance source
# stickiness
stick-table type ip size 50k expire 30m
stick on src
# tuning options
timeout connect 30s
timeout server 30s
server zz_Ubuntu01_SSL 192.168.112.111:443
# statistics are DISABLED
I'm grateful for any hint!
Best regards,
Philipp
What webserver are you using?
Why are you using TCP mode on the SSL frontend and backend?
Edit: derp.. That's how you're meant to do it... Referring to this https://www.haproxy.com/blog/enhanced-ssl-load-balancing-with-server-name-indication-sni-tls-extension/. Config looks ok.
Are you getting any errors on the webserver?
Edit 2: Have you tried the setup as per "Choose a server using SNI: aka SSL routing" from the link above?
@cmdr.adama: Thanks for your reply .
I'm using Apache webservers. I'm going to check the error log of the server, maybe I can find some hint.
How can I implement the second way "Choose a server using SNI: aka SSL routing" in combination with OpnSense when it's not possible to edit the config file directly?
Is there any way to get a detail debug log out of HaProxy running on opnSense?
cmdr.adama thank you very much
Quote from: hwsweng on May 12, 2020, 10:49:09 PM
@cmdr.adama: Thanks for your reply .
I'm using Apache webservers. I'm going to check the error log of the server, maybe I can find some hint.
How can I implement the second way "Choose a server using SNI: aka SSL routing" in combination with OpnSense when it's not possible to edit the config file directly?
Is there any way to get a detail debug log out of HaProxy running on opnSense?
No probs,
You'll need to make the necessary changes in the GUI which will apply to the config. Did you find anything in the Apache logs? I suspect you'll get SSL errors
I have checked the apache logs and the server isn't even reached when I try to use haproxy for the SSL traffic.
In the haproxy log I get: haproxy[34872]: http_dispatcher_SSL http_dispatcher_SSL/<NOSRV> -1/-1/0 0 SC 1/1/0/0/0 0/0
for me it looks like that the rules aren't taken effect ???
I have updated haproxy and the entire opnSense to the newest version in the meantime but no change regarding my problem.
Now I have played a bit with the configuration and haproxy is selecting the right backend (but I have no clue wich setting change did this effect :-[ ). One step ahead...
The backend server is getting connected now, but I get on my apache the error:
[ssl:info] [pid 23392] [client 192.168.xx.xxx:23800] AH01964: Connection to child 2 established (server localhost:443)
[ssl:info] [pid 23392] [client 192.168.xx.xxx:23800] AH02008: SSL library error 1 in handshake (server localhost:443)
[ssl:info] [pid 23392] SSL Library Error: error:1408F10B:SSL routines:ssl3_get_record:wrong version number
[ssl:info] [pid 23392] [client 192.168.xx.xxx:23800] AH01998: Connection closed to child 2 with abortive shutdown (server localhost:443)
The client IP in the log 192.168.xx.xxx is the local IP of my opnSense, maybe this have to do something with the proxy settings in the haproxy backend?
EDIT:
To check this theory I have disabled the proxy in the backendpool and now it works :D
But one thing is an open question: Is it possible to make the request client IP transparent to the webserver, at the moment there is only the opnsense IP in the apache Logs for the request of all clients (thats pretty clear to me why this is the case, but is there a possibility for a work around?).
Hey @hwsweng
as I'm new to the plugin - I would be very interested in the working configuration.
If it is possible with the relevant fields from the plugin.
This could be very helpfull,
Best,
Bernd
Quote from: hwsweng on June 17, 2020, 10:32:13 PM
EDIT:
To check this theory I have disabled the proxy in the backendpool and now it works :D
But one thing is an open question: Is it possible to make the request client IP transparent to the webserver, at the moment there is only the opnsense IP in the apache Logs for the request of all clients (thats pretty clear to me why this is the case, but is there a possibility for a work around?).
Hi, how can you disable the proxy in the backendpool to make it works
Quote from: hwsweng on June 17, 2020, 10:32:13 PM
Now I have played a bit with the configuration and haproxy is selecting the right backend (but I have no clue wich setting change did this effect :-[ ). One step ahead...
I am having this same issue, would you be will to either post or pm your config file?