24.1.3_1 - HAProxy, Gateway group - Configuration Assistance

[Nebulis01]

This configuration is no longer functional after upgrading from 23.7.12 to 24.1.x – I assume I've misconfigured something and would like assistance if anyone is able and willing.

I have two WAN connections in a gateway group one with a static IPv4 address and a number of virtual IPs,  the other connection has a single dhcp address.

The gateway group has the WAN with the static block configured with a priority of 1 and the 'upstream gateway' box checked.

The 'gateway switching' option is check in System > Settings > General

I use a port forward rule to pass traffic incoming on :443 to 127.0.0.1:7443 for HAProxy.

There is a floating rule for incoming traffic on either WAN interface to allow 127.0.0.1:7443

I see traffic pass the firewall and redirect appropriately:

Code Select Expand

2024-03-09T18:07:37-08:00 Informational filterlog 87,,,63615a3f759453d7effd576cf389d1ba,igc0,match,pass,in,4,0x0,,244,54321,0,none,6,tcp,44,<sourceip>,127.0.0.1,51684,7443,0,S,2650071847,,65535,,mss

2024-03-09T18:07:37-08:00 Informational filterlog 36,,,0,igc0,match,rdr,in,4,0x0,,244,54321,0,none,6,tcp,44,<sourceip>,<dstip>,51684,443,0,S,2650071847,,65535,,mss

2024-03-09T18:06:53-08:00 Informational filterlog 87,,,63615a3f759453d7effd576cf389d1ba,igc0,match,pass,in,4,0x0,,50,0,0,none,6,tcp,60,<sourceip>,127.0.0.1,48642,7443,0,S,1531076519,,65535,,mss;sackOK;TS;nop;wscale

2024-03-09T18:06:53-08:00 Informational filterlog 36,,,0,igc0,match,rdr,in,4,0x0,,50,0,0,none,6,tcp,60,<sourceip>,<dstip>,48642,443,0,S,1531076519,,65535,,mss;sackOK;TS;nop;wscale

I don't see the traffic in HA Proxy when the primary gateway is in use – If change the gateway to the secondary, everything functions (the traffic still comes in across the primary gateway but the haproxy configuration works, I'm at a loss)

The only item I see in the haproxy log is that the back-end pool stopped, but I expect this restarting the service to troubleshoot

The HA Proxy config file is here:

Code Select Expand


global
    uid                         80
    gid                         80
    chroot                      /var/haproxy
    daemon
    stats                       socket /var/run/haproxy.socket group proxy mode 775 level admin expose-fd listeners
    nbthread                    1
    hard-stop-after             60s
    no strict-limits
    httpclient.resolvers.prefer   ipv4
    tune.ssl.default-dh-param   2048
    spread-checks               2
    tune.bufsize                16384
    tune.lua.maxmem             0
    log                         /var/run/log local0 debug
    lua-prepend-path            /tmp/haproxy/lua/?.lua

defaults
    log     global
    option redispatch -1
    timeout client 30s
    timeout connect 30s
    timeout server 30s
    retries 3
    default-server init-addr last,libc

# autogenerated entries for ACLs

# autogenerated entries for config in backends/frontends

# autogenerated entries for stats




# Frontend: blueiris-public-pool (HTTPS for BlueIris listen on 7443)
frontend blueiris-public-pool
    http-response set-header Strict-Transport-Security "max-age=15768000"
    bind 127.0.0.1:7443 name 127.0.0.1:7443 ssl prefer-client-ciphers ssl-min-ver TLSv1.2 ciphers ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-CHACHA20-POLY1305:ECDHE-RSA-CHACHA20-POLY1305:ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES256-SHA384:ECDHE-ECDSA-AES128-SHA256 ciphersuites TLS_AES_128_GCM_SHA256:TLS_AES_256_GCM_SHA384:TLS_CHACHA20_POLY1305_SHA256 alpn h2,http/1.1 crt-list /tmp/haproxy/ssl/6242b2814feb11.18910711.certlist
    mode http
    option http-keep-alive
    default_backend blueiris-backend-pool
    option forwardfor

    # logging options
    option httplog

# Backend: blueiris-backend-pool ()
backend blueiris-backend-pool
    # health checking is DISABLED
    mode http
    balance source
    # stickiness
    stick-table type ip size 50k expire 30m 
    stick on src
    http-reuse safe
    server blueiris 192.168.25.50:81

# Backend (DISABLED): acme_challenge_backend (Added by ACME Client plugin)



listen local_statistics
    bind            127.0.0.1:8822
    mode            http
    stats uri       /haproxy?stats
    stats realm     HAProxy\ statistics
    stats admin     if TRUE

# remote statistics are DISABLED



I see that haproxy is listening on 127.0.0.1:7443

Code Select Expand

root@OPNsense:~ # sockstat | grep haproxy
www      haproxy    39769 5  tcp4   127.0.0.1:7443        *:*
www      haproxy    39769 6  tcp4   127.0.0.1:8822        *:*
www      haproxy    39769 8  stream /var/run/haproxy.socket.41133.tmp
root     syslog-ng  24034 24 dgram  /var/haproxy/var/run/log

Anyone have any idea where I should be looking for additional logging or what's gone sideways?

[Nebulis01]

As an update - this issue resolved between 24.1.3_1 and 24.1.10_3

With no configuration changes the configuration is functional once again.