HAProxy configuration with multiple WANs

[mprajescu]

I’ve been struggling with this issue for quite some time and cannot figure it out.
 For the sake of confidentiality of the public IPs I will use different IP address for this configuration example.

This is my current setup:

The firewall has 3 WAN interfaces:

WAN
Address: 192.168.43.253/26
Gateway: 192.168.43.193
Subnet Mask: 255.255.255.192

WAN2
Address: 192.168.44.220/26
Gateway: 192.168.44.193
Subnet Mask: 255.255.255.192

WAN3
Address: 192.168.44.221/26
Gateway: 192.168.44.193
Subnet Mask: 255.255.255.192


In the firewall, the WAN2 is configured with the GW directly and the WAN3 has the GW field set “Default” so it will choose the automatic one.

Firewall is accessible from outside, on a different port, 444, with a floating rule, on destination WAN Address, to port 444 to allow incoming connections.

On WAN, I have done a port forward to the mail server for 25,465,587,993, 80 and 443 due to the services that it requires. I do not need to do 1:1 to the whole server so doing a Port Forward does the job.

On WAN2, I have done a port forward to 80 and 443 to an internal web server, which handles the requests perfectly fine.

I’ve setup a floating rule, for interface WAN3, as follows:

Interface: WAN3
Direction: in
TCP/IP version: IPv4
Protocol: TCP/UDP
Source: Any
Destination: WAN3 Address
Destination Port Range:
- from: HTTP
- to: HTTP
Log: Ticked the Log packets
Description: Enable HTTP on WAN3

I’ve cloned the rule and also added HTTPS ports and updated the Description filed to "HTTPS" instead of "HTTP".

Here comes the issue. When I setup the floating rule, and navigate to Firewall-> Rules -> WAN3, it is saying
 “no WAN3 rules are currently defined. All incoming connections on this interface will be blocked until you add a pass rule. Exceptions for automatically generated rules may apply”

Does this mean that the floating rule is not applying correctly?


I’ve checked the firewall log under live view, and the floating rule allows traffic in.

The HAProxy configuration is as follows:


Real Servers:

Advanced mode: unchecked
Name: ITEC_Portal
Description: Portal
FQDN or IP: 192.168.10.8
Port: 8443
Mode: Active
SSL: Checked
Verify SSL Certificate: Unchecked
SSL Verify CA: Nothing Selected

Backend Pools:

Name: Backend_ITEC_Portal
Description: Backend_Portal
Mode: HTTP (Layer 7) [default]
Balancing Algorithm: Source-IP Hash [default]
Servers: ITEC_Portal
Enable Health Checking: Unchecked
Enable HTTP/2: Checked
HTTP/2 without TLS: Unchecked
Advertise Protocols (ALPN): HTTP/2, HTTP/1.1
Persistence type: Stick-table persistence [default]
Table Type: Source-IP [default]
Stored data types: Nothing Selected
Cookie name: blank
Cookie length: blank
Basic Authentication Enable: unchecked
Allowed Users: Nothing Selected
Allowed Groups: Nothing Selected
Retries: blank
Select Rules: blank
Select Error Messages: Nothing selected

Conditions:
Name: Condition_ITEC_Portal
Description: Portal
Condition type: Host starts with
Negate condition: Unchecked
Host Prefix: portal.example.com

Rules:

Name: Rule_ITEC_Portal
Description: Portal
Test type: IF [default]
Select conditions: Condition_ITEC_Portal
Logical operator for conditions: AND [default]
Execute function: Use specified Backend Pool
Use backend pool: Backend_ITEC_Portal


Public Services:

Enabled: Checked
Name: HTTPS_WAN3_Public_Service
Description: HTTPS listener
Listen Addresses: 127.0.0.1:443, 192.168.10.1:443, 192.168.44.221:443
Type: HTTP/HTTPS (SSL offloading) [default]
Default Backend Pool: None
Enable SSL offloading: Checked
Certificates: Official Public Wildcard Certificate
Default certificate: Official Public Wildcard Certificate
Enable Advanced settings: unchecked
Client Certificate Auth Enable: Unchecked
Verification: None
Certificate Authorities: Nothing Selected
Certificate Revocation Lists: Nothing Selected
Enable HTTP/2: Checked
HTTP/2 without TLS: Unchecked
Advertise Protocols (ALPN): HTTP/2, HTTP/1.1
X-Forwarded-For header: Checked
Basic Authentication Enable: Unchecked
Allowed Users: Nothing Selected
Allowed Groups: Nothing Selected
Max Connections: blank
Detailed Logging: Checked
Table type: None
Stored data types: Nothing selected
Rules: Rule_ITEC_Portal
Select Error Messages: Nothing Selected


Enabled: Checked
Name: HTTP_WAN3_Public_Service
Description: HTTPS listener
Listen Addresses: 127.0.0.1:80, 192.168.10.1:80, 192.168.44.221:80
Type: HTTP/HTTPS (SSL offloading) [default]
Default Backend Pool: None
Enable SSL offloading: Unchecked
Enable HTTP/2: Checked
HTTP/2 without TLS: Unchecked
Advertise Protocols (ALPN): HTTP/2, HTTP/1.1
X-Forwarded-For header: Checked
Basic Authentication Enable: Unchecked
Allowed Users: Nothing Selected
Allowed Groups: Nothing Selected
Max Connections: blank
Detailed Logging: Checked
Table type: None
Stored data types: Nothing selected

The HAProxy service is started and remains started. I checked in the lobby and also on the HAProxy page, the green running button is on top of the page.

The issue is that I can access the websites if I am trying to get to them from the internal network.
I don't see anything in the logs when I try to access from the outside.

Any help is appreciated.

I tried to set only the listening address to the external IP address
I tried to do a binding to internal IP address (192.168.10.2) still not working.
I tried removing the offloading of the SSL, I still don't get from the outside to the inside.

[mprajescu]

After a lot of digging around it was the data center ISP routing issue.

I have changed the gateway and setup WAN2 normally with the dedicated gateway and changed HAProxy to listen on WAN2 Public IP Address with two public services one on port 80 and one on port 443.

Everything resumed and worked normally.

The only issue I seem to face now is how to create 2 public services in HAProxy and listen to the same IP and same Port but one service will encrypt traffic and one shouldn’t encrypt and pass through for the server to encrypt the traffic.

Also, having dedicated and paid wildcard certificates help.
To reduce cost of operating I wanted to enable Let’s Encrypt plug-in.
The let’s encrypt plug-in saves the HAProxy configuration and the HTTP challenge is setup correctly, however it doesn’t create the Trusted certificate at all and nothing is showing in the log page.

Any pointers are welcomed.