HAproxy not starting - looking for help [SOLVED]

[rfonteyn]

Problem Description
First off, you can consider me a newbie.

I have an opnsense install, running on proxmox.
I tried to set up HAproxy and followed this guide: https://forum.opnsense.org/index.php?topic=23339.0

When I do a Test Syntax, I get

Your HAProxy config contains no errors.

. But still, HA fails to start.
lastlog output

Code Select Expand

<13>1 2025-06-11T09:53:49+02:00 gauloises.fonteyn.info root 40965 - [meta sequenceId="3714"] /usr/local/etc/rc.d/haproxy: WARNING: failed to start haproxy

Where can check for further troubleshooting. And possible a solution?

OPNsense

Type   opnsense   
Version   25.1.7_4   
Architecture   amd64   
Commit   86e8d5f88   
Mirror   https://pkg.opnsense.org/FreeBSD:14:amd64/25.1   
Repositories   OPNsense (Priority: 11)   
Updated on   Mon May 26 22:07:53 CEST 2025   
Checked on   Wed Jun 11 09:33:31 CEST 2025

HAproxy Config

Code Select Expand

#
# Automatically generated configuration.
# Do not edit this file manually.
#

global
    uid                         80
    gid                         80
    chroot                      /var/haproxy
    daemon
    stats                       socket /var/run/haproxy.socket group proxy mode 775 level admin
    nbthread                    4
    hard-stop-after             60s
    no strict-limits
    maxconn                     10000
    ocsp-update.mindelay 300
    ocsp-update.maxdelay 3600
    httpclient.resolvers.prefer   ipv4
    tune.ssl.default-dh-param   4096
    spread-checks               2
    tune.bufsize                16384
    tune.lua.maxmem             0
    log                         /var/run/log local0 debug
    lua-prepend-path            /tmp/haproxy/lua/?.lua

defaults
    log     global
    option redispatch -1
    maxconn 5000
    timeout client 30s
    timeout connect 30s
    timeout server 30s
    retries 3
    default-server init-addr last,libc

# autogenerated entries for ACLs


# autogenerated entries for config in backends/frontends

# autogenerated entries for stats




# Frontend: 0_SNI_frontend (Listening on 0.0.0.0:80, 0.0.0.0:443)
frontend 0_SNI_frontend
    bind 0.0.0.0:443 name 0.0.0.0:443
    bind 0.0.0.0:80 name 0.0.0.0:80
    mode tcp
    default_backend SSL_backend

    # logging options

# Frontend: 1_HTTP_frontend (Listening on 127.4.4.3:80)
frontend 1_HTTP_frontend
    bind 127.4.4.3:80 name 127.4.4.3:80 accept-proxy
    mode http
    option http-keep-alive
    option forwardfor

    # logging options
    # ACL: NoSSL_condition
    acl acl_6842b26eea5709.95826104 ssl_fc

    # ACTION: HTTPtoHTTPS_rule
    http-request redirect scheme https code 301 if !acl_6842b26eea5709.95826104

# Frontend: 1_HTTPS_frontend (Listening on 127.4.4.3:443)
frontend 1_HTTPS_frontend
    http-response set-header Strict-Transport-Security "max-age=15768000; includeSubDomains; preload"
    bind 127.0.0.3:443 name 127.0.0.3:443 accept-proxy ssl curves secp384r1  no-sslv3 no-tlsv10 no-tlsv11 no-tls-tickets ciphers ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-CHACHA20-POLY1305:ECDHE-RSA-CHACHA20-POLY1305:ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES256-SHA384:ECDHE-ECDSA-AES128-SHA256 ciphersuites TLS_AES_128_GCM_SHA256:TLS_AES_256_GCM_SHA384:TLS_CHACHA20_POLY1305_SHA256 alpn h2,http/1.1 crt-list /tmp/haproxy/ssl/6842b6333c6573.77210662.certlist
    mode http
    option http-keep-alive

    # logging options

    # ACTION: PUBLIC_SUBDOMAINS_rule
    # NOTE: actions with no ACLs/conditions will always match
    use_backend %[req.hdr(host),lower,map_dom(/tmp/haproxy/mapfiles/6842b30d684377.19778304.txt)]

# Backend: SSL_backend ()
backend SSL_backend
    # health checking is DISABLED
    mode tcp
    balance source
    # stickiness
    stick-table type ip size 50k expire 30m 
    stick on src
    server proxmox 10.82.162.254:443 ssl alpn h2,http/1.1 verify required ca-file /usr/local/etc/ssl/cert.pem send-proxy-v2 check-send-proxy

# Backend: mindbodyreconnect ()
backend mindbodyreconnect
    # health checking is DISABLED
    mode http
    balance source
    # stickiness
    stick-table type ip size 50k expire 30m 
    stick on src
    http-reuse safe
    server www.mindbodyreconnect.be 10.82.162.173:80

# Backend: courses ()
backend courses
    # health checking is DISABLED
    mode http
    balance source
    # stickiness
    stick-table type ip size 50k expire 30m 
    stick on src
    http-reuse safe
    server courses 10.82.162.174:80

# Backend: proxmox ()
backend proxmox
    # health checking is DISABLED
    mode http
    balance source
    # stickiness
    stick-table type ip size 50k expire 30m 
    stick on src
    http-reuse safe
    server proxmox 10.82.162.254:443 ssl alpn h2,http/1.1 verify required ca-file /usr/local/etc/ssl/cert.pem



# statistics are DISABLED



[cookiemonster]

Stab in the dark without your overall setup information.
You are trying to bind haproxy to ports 80 and 443. Did you move the OPN GUI to alternative ports to avoid the conflicts ?

[rfonteyn]

Hi,

Yes I did, as per the instructions on the linked Tutorial.

If you need more specific, relevant info regarding my setup or issue, please ask and I will provide whatever I can.

[cookiemonster]

Whole network topology setup. Check out the post from meyergru for considerations when virtualising. That is for an overall check.
For instance you have from your post OPN running as a VM on Proxmox. Then your first  backend is "server proxmox 10.82.162.254:443" so you have to have a route. Can your OPN and importantly haproxy get to that ip?

[rfonteyn]

Whole network topology setup. Check out the post from meyergru for considerations when virtualising. That is for an overall check.
For instance you have from your post OPN running as a VM on Proxmox. Then your first  backend is "server proxmox 10.82.162.254:443" so you have to have a route. Can your OPN and importantly haproxy get to that ip?

Currently not behind my PC, but yes. That subnet is connected on the LAN interface.

OPNsense LAN interface op is 10.82.162.1/24. And I can reach the backend servers locally and through tailscale (routed over OPNsense)


Logical design is very simple

Internet -- Wan interface -- OPNsense -- LAN interface -- with backend servers

[rfonteyn]

Additionally, the virtualisation works fine and has been running for a few weeks.

Ddns, acme, ids (not yet IPS), DHCP, unbound are all working fine.

Direct rules with port forwarding to the backend worked fine while testing. But has been disabled again, as I don't want internal servers directly exposed to the internet.

Where does haproxy writes it's logging? Because all I can find is the message in the original post.

[cookiemonster]

Logical design is very simple

Internet -- Wan interface -- OPNsense -- LAN interface -- with backend servers

That helps but the physical design is different, right? with Proxmox and its own networking in the setup; but not in the picture.

Haproxy > Maintenance Servers. Are all showing Active = 1 ?

Log file is Haproxy > Log file. But I find it difficult to work with because is noisy so hard to spot the problem.

Suggest to attempt to start it and go straight to it, chose Debug and look for clues.
Edit: logs are in  /var/log/haproxy/
latest.log is the current and rotated ones alongside it.

[rfonteyn]

...
That helps but the physical design is different, right? with Proxmox and its own networking in the setup; but not in the picture.

Haproxy > Maintenance Servers. Are all showing Active = 1 ?

Log file is Haproxy > Log file. But I find it difficult to work with because is noisy so hard to spot the problem.

Suggest to attempt to start it and go straight to it, chose Debug and look for clues.
Edit: logs are in  /var/log/haproxy/
latest.log is the current and rotated ones alongside it.

First: Thanks for helping me out.

I'll upload a drawing of the physical and logical layout .

physical
- OPNsense runs on pveFW1 OR pveFW2 (redundancy based on proxmox HA). So no HA configured in OPNsense
- VLANS are handled by proxmox. Passed on to OPNsense based on the respective interface. So OPNsense has no notion of the vlans.


logical

There's a lot more on the drawing than already in use. This is a learning lab under construction.

Haproxy > Maintenance Servers. Are all showing Active = 1 ? >> "No results found!"
Haproxy > Log file is empty
(HAproxy > Settings > Settings > Logging  "Filter syslog level" is set to debug)

[cookiemonster]

Haproxy > Maintenance Servers. Are all showing Active = 1 ? >> "No results found!"

This could be the problem. As I found very recently, if a server defined in HAProxy isn't reachable, it was unable to start. But there were clues to that effect either in haproxy log or in OPN's (I can't remember the steps I took to narrow it down).
But then I was lucky that I remembered I had updated and restarted another proxmox node. So I looked and realised a new server (real server to haproxy) was not set to autostart.
In your case and the reason I suggest to check your topology, is to verify that the real servers are reachable by haproxy.
Also check if you have set them up with ip address or by name (fqdn), in which case the name needs to be resolved by OPN so that haproxy can use its ip.

[rfonteyn]

Haproxy > Maintenance Servers. Are all showing Active = 1 ? >> "No results found!"

This could be the problem. As I found very recently, if a server defined in HAProxy isn't reachable, it was unable to start. But there were clues to that effect either in haproxy log or in OPN's (I can't remember the steps I took to narrow it down).
But then I was lucky that I remembered I had updated and restarted another proxmox node. So I looked and realised a new server (real server to haproxy) was not set to autostart.
In your case and the reason I suggest to check your topology, is to verify that the real servers are reachable by haproxy.
Also check if you have set them up with ip address or by name (fqdn), in which case the name needs to be resolved by OPN so that haproxy can use its ip.

Seems a bit counterproductive, that the whole system stays down if one real server is not reachable.

Anyway, I checked from OPNsense > interfaces > Diagnostics > ping, and from CLI, your recommendation and all three servers were up and reachable. on the configured names. to simplify further troubeshooting, I've removed  the config for two of the three for now and retried starting HAproxy. Still the same result.

[rfonteyn]

I managed to get some more hints as to where my problem is situated:

Code Select Expand

root@gauloises:/etc # haproxy -d -f /usr/local/etc/haproxy.conf
Available polling systems :
     kqueue : pref=300,  test result OK
       poll : pref=200,  test result OK
     select : pref=150,  test result FAILED
Total: 3 (2 usable), will use kqueue.

Available filters :
[BWLIM] bwlim-in
[BWLIM] bwlim-out
[CACHE] cache
[COMP] compression
[FCGI] fcgi-app
[SPOE] spoe
[TRACE] trace
Using kqueue() as the polling mechanism.
[NOTICE]   (45542) : haproxy version is 3.0.10-346eb4f
[NOTICE]   (45542) : path to executable is /usr/local/sbin/haproxy
[ALERT]    (45542) : Binding [/usr/local/etc/haproxy.conf:72] for frontend 1_HTTPS_frontend: protocol tcpv4: cannot bind socket (Can't assign requested address) for [127.0.0.3:443].
[ALERT]    (45542) : [haproxy.main()] Some protocols failed to start their listeners! Exiting.
root@gauloises:/etc #

Now to find out why it can't bind the socket. And why it is trying to use frontend 127.0.0.3, where the configured frondends are 127.4.4.3

[Patrick M. Hausen]

127.4.4.3 is still in the loopback network of 127/8. What are you trying to achieve with that?

[rfonteyn]

I followed a tutorial from the the forum here, which was using this address as a virtual IP.

But after finally finding the root cause, I managed to solve my issue. I changed those frontends to the actual loopback address 127.0.0.1 and now at least haproxy starts and is redirecting me.

[cookiemonster]

That setup works fine. Maybe you didn't create the VIP on the loopback address space first ?