Messages

121

Tutorials and FAQs / Re: Tutorial 2022/06: HAProxy + Let's Encrypt Wildcard Certificates + 100% A+ Rating

« on: July 04, 2022, 11:02:05 pm »

So, I conclude that the certificate is O.K., I get through to 1_HTTP_Frontend (otherwise there would be no 503 when I use an unmapped URL). I seems like most of the time, the frontend does not connect to the backend (server) - it does sometimes, however.

I am at a loss what causes this because I also verified that OpnSense can reach BLOB server at 192.168.10.3. It also has a Let's Encrypt certificate on ssl.xyz.de, because before, I just port-forwarded port 443. I can get data via 'curl --connect-to 192.168.10.3:443 https://ssl.xyz.de' from it.

Code: [Select]

server BLOB_server 192.168.10.3 ssl sni str(ssl.xyz.de) verify none resolve-prefer ipv4
1. You have to include the server port in the config of the server, obviously...
2. Remove the SNI string in the server config, not neccessary at all. I also DID NOT use it in my tutorial, so why are you using it?
3. Remove the resolve-prefer directive. Again, I did not use it in my tutorial...

122

Tutorials and FAQs / Re: Tutorial 2022/06: HAProxy + Let's Encrypt Wildcard Certificates + 100% A+ Rating

« on: June 27, 2022, 04:08:58 pm »

I could just call up everything by subdomain.domain.com locally.

This is because your domains resolve to cloudflares IP and not your own public IP.

Also with regards to the map file for the local subdomanins. The format should look like this?

local loca_backend

plex plex_backend

Should there be a space in between?

https://www.haproxy.com/documentation/hapee/latest/configuration/map-files/syntax/

Last questions. Is it possible to use the generated Cert for both truenas and opnsense instead of the self signed one? would it break the config? Also is there a specifi HAP sub, or should i just post in the General section?

Shouldn't break anything but isn't neccessary.
https://discourse.haproxy.org/
or
https://forum.opnsense.org/index.php?board=28.0

123

Tutorials and FAQs / Re: Tutorial 2022/06: HAProxy + Let's Encrypt Wildcard Certificates + 100% A+ Rating

« on: June 21, 2022, 10:58:08 am »

Heres and output of my config file, i have some disabled stuff in the config file since i turn on and off stuff for testing. Also renamed some stuff, truenas is just the Physical server with the IP. (myloopbackip) is the virutal address.

Code: [Select]

#
# Automatically generated configuration.
# Do not edit this file manually.
#

#
# NOTE: HAProxy is currently DISABLED
#
global
    uid                         80
    gid                         80
    chroot                      /var/haproxy
    daemon
    stats                       socket /var/run/haproxy.socket group proxy mode 775 level admin
    nbproc                      1
    nbthread                    4
    hard-stop-after             60s
    no strict-limits
    maxconn                     10000
    tune.ssl.default-dh-param   4096
    spread-checks               2
    tune.bufsize                16384
    tune.lua.maxmem             0
    log                         /var/run/log local0 info
    lua-prepend-path            /tmp/haproxy/lua/?.lua

defaults
    log     global
    option redispatch -1
    maxconn 5000
    timeout client 30s
    timeout connect 30s
    timeout server 30s
    retries 3
    default-server init-addr libc,last
    default-server maxconn 5000

# autogenerated entries for ACLs


# autogenerated entries for config in backends/frontends

# autogenerated entries for stats




# Frontend: SNI_frontend (Listening on ip:80 / ip:443)
frontend SNI_frontend
    bind 0.0.0.0:80 name 0.0.0.0:80
    bind 0.0.0.0:443 name 0.0.0.0:443
    bind :::80 name :::80
    bind :::443 name :::443
    mode tcp
    default_backend SSL_Backend
    # tuning options
    timeout client 30s

    # logging options

# Frontend: HTTP_frontend (Listening on 127.10.20.5)
frontend HTTP_frontend
    bind (myloopbackip):80 name (myloopbackip):80 accept-proxy
    bind :::80 name :::80 accept-proxy
    mode http
    option http-keep-alive
    option forwardfor
    # tuning options
    timeout client 30s

    # logging options
    # ACL: NoSSL_Condition
    acl acl_62aa8dcf894a87.42381056 ssl_fc

    # ACTION: HTTPtoHTTPS
    http-request redirect scheme https code 301 if !acl_62aa8dcf894a87.42381056

# Frontend: HTTPS_frontend (Listening on (myloopbackip))
frontend HTTPS_frontend
    http-response set-header Strict-Transport-Security "max-age=63072000; includeSubDomains; preload"
    bind (myloopbackip):443 name (myloopbackip):443 accept-proxy ssl curves secp384r1  no-sslv3 no-tlsv10 no-tlsv11 no-tls-tickets ciphers ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-CHACHA20-POLY1305:ECDHE-RSA-CHACHA20-POLY1305:DHE-RSA-AES256-GCM-SHA384 ciphersuites TLS_AES_256_GCM_SHA384:TLS_CHACHA20_POLY1305_SHA256 alpn h2,http/1.1 crt-list /tmp/haproxy/ssl/62aad04a028639.71957640.certlist
    bind :::443 name :::443 accept-proxy ssl curves secp384r1  no-sslv3 no-tlsv10 no-tlsv11 no-tls-tickets ciphers ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-CHACHA20-POLY1305:ECDHE-RSA-CHACHA20-POLY1305:DHE-RSA-AES256-GCM-SHA384 ciphersuites TLS_AES_256_GCM_SHA384:TLS_CHACHA20_POLY1305_SHA256 alpn h2,http/1.1 crt-list /tmp/haproxy/ssl/62aad04a028639.71957640.certlist
    mode http
    option http-keep-alive
    option forwardfor
    # tuning options
    timeout client 30s

    # logging options

    # ACTION: Public_subdomains_map_rule
    # NOTE: actions with no ACLs/conditions will always match
    use_backend %[req.hdr(host),lower,map_dom(/tmp/haproxy/mapfiles/62aa8e31993357.88056717.txt)]
    # WARNING: pass through options below this line
    http-request set-header X-Forwarded-Proto https if { ssl_fc }

# Backend: SSL_Backend ()
backend SSL_Backend
    # health checking is DISABLED
    mode tcp
    balance source
    # stickiness
    stick-table type ip size 50k expire 30m 
    stick on src
    # tuning options
    timeout connect 30s
    timeout server 30s
    server SSL_Server (myloopbackip) send-proxy-v2 check-send-proxy

# Backend (DISABLED): router_Backend (router Backend)

# Backend: truenas_Backend (truenas Backend)
backend truenas_Backend
    # health checking is DISABLED
    mode http
    balance source
    # stickiness
    stick-table type ip size 50k expire 30m 
    stick on src
    # tuning options
    timeout connect 30s
    timeout server 30s
    # WARNING: pass through options below this line
    http-request set-header X-Forwarded-Proto https if { ssl_fc }
    http-reuse safe
    server truenas truenasip:443 ssl sni str(truenas) verify none send-proxy-v2 check-send-proxy

# Backend: plex_backend (plex Backend)
backend plex_backend
    # health checking is DISABLED
    mode http
    balance source
    # stickiness
    stick-table type ip size 50k expire 30m 
    stick on src
    # tuning options
    timeout connect 30s
    timeout server 30s
    http-reuse safe
    server Plex truenasip:32400 send-proxy-v2 check-send-proxy


This is my map file:

Code: [Select]

# public access domains
truenas truenas_backend
plex plex_backend


1. The map file is case sensitive. Fix it.

2. Remove the "send-proxy-v2 check-send-proxy" directives from the backends of your actual services. These two options are only necessary on the "SSL_backend".

3. Why do you have the sni setting configured in your truenas real server? that shouldn't be necessary at all. Also remove the "send-proxy-v2 check-send-proxy" directives from the backend (see 1.).

Code: [Select]

server truenas truenasip:443 ssl sni str(truenas) verify none send-proxy-v2 check-send-proxy
4. Your "HTTP_frontend" and "HTTPS_frontend" should ONLY be listening to your SSL_server IP address. Not to "0.0.0.0:0" or "::::0". Just think about it and take a look at my diagram in the first post... You should quickly figure that this doesn't make any sense.

124

Tutorials and FAQs / Re: Tutorial 2022/06: HAProxy + Let's Encrypt Wildcard Certificates + 100% A+ Rating

« on: June 21, 2022, 10:49:24 am »

I've followed the article and able to setup few subdomain to internal machines in my network.

I have dumb questions... when I created new real server and backend server for one my Synology packages, I initially used HTTPS port and received 400 bad request error The plain HTTP request was sent to HTTPS port

When I changed the port in real server settings to HTTP request, everything worked fine and I am able to access the internal server with SSL certificate.

Why?

FAQ - Do I need to enable "SSL" in the Real Server configuration of a service?

Thanks for NOT reading my first post properly. 
I explained everything there very well.

125

Tutorials and FAQs / Re: Tutorial 2022/06: HAProxy + Let's Encrypt Wildcard Certificates + 100% A+ Rating

« on: June 17, 2022, 10:03:40 pm »

Hi! First off, i want to thank you for the detailed guide you posted. I am new to HAProxy and have some questions regarding some configs, do you still reply to this post? or should i ask or open another thread somewhere? Thanks!

As long as the question is related to THIS tutorial then feel free to ask!

Otherwise please ask here: https://forum.opnsense.org/index.php?board=28.0

126

Tutorials and FAQs / Re: Tutorial 2022/06: HAProxy + Let's Encrypt Wildcard Certificates + 100% A+ Rating

« on: June 15, 2022, 10:23:09 pm »

When I try to access from internal LAN (IP address or Name), I get the following error:

• Firefox:SSL_ERROR_RX_RECORD_TOO_LONG
• Edge:ERR_SSL_PROTOCOL_ERROR

This error usually means that you tried to access a service using HTTPS that only supports HTTP.

Does this mean I need an additional vhost config for the Nextcloud? One for :80 and one for :443? But the certificate lies on a different machine (opnsense). How can I point it there? Or am I missing something?
By deleting the "overwirte" codes in the nextcloud config I at least got access in my internal LAN. I assume the HAProxy config is correct and I need to make corrections in the nextcloud config.

Code: [Select]

    server Nextcloud 192.168.10.40:443 ssl verify noneThis line in your HAProxy config indicates that you are accessing your nextcloud webinterface from your local network using "https://192.168.10.40".
If the above is not true and you are accessing it using "http://192.168.10.40" instead then you should have figured your error after reading this sentence.

IF you can already access nextcloud from within your local using its Local_IP:Port then you won't have to change any settings at nextcloud.

Also... Please do yourself a favour and read about what a reverse proxy is, what it does and how it works.
No offense but I think you don't really know what you are doing here.

127

Tutorials and FAQs / Re: Tutorial 2022/06: HAProxy + Let's Encrypt Wildcard Certificates + 100% A+ Rating

« on: June 15, 2022, 03:25:18 pm »

Since you are forcing HAProxy to use TLS 1.3. (As you use no-sslv3 no-tlsv10 no-tlsv11 no-tls-tickets no-tlsv12)
There might be issue taking out TLS_AES_128_GCM_SHA256.

According to RFC 8446, S9.1

A TLS-compliant application MUST implement the TLS_AES_128_GCM_SHA256 [GCM] cipher suite and SHOULD implement the TLS_AES_256_GCM_SHA384 [GCM] and TLS_CHACHA20_POLY1305_SHA256 [RFC8439] cipher suites (see Appendix B.4).

Well, in order to get an 100% A+ rating at SSL Labs it is mandatory to remove any ciphers below or equal to 128 bit. This is why I haven't included them in my cipher list and cipher suites.
So unless you have really old devices with outdated browsers I would suggest to only use ciphers with a strength of 256 bit or above.

128

Tutorials and FAQs / Re: Tutorial 2022/06: HAProxy + Let's Encrypt Wildcard Certificates + 100% A+ Rating

« on: June 15, 2022, 03:15:28 pm »

When I try to access from internal LAN (IP address or Name), I get the following error:

• Firefox:SSL_ERROR_RX_RECORD_TOO_LONG
• Edge:ERR_SSL_PROTOCOL_ERROR

This error usually means that you tried to access a service using HTTPS that only supports HTTP.

internal LAN (IP address or Name)

What do you mean by that? You should use the FQDN and not the local hostname / IP in order to use the reverse proxy (HAProxy).

If I enter http://IP_ADRESS then it opens the Nextcloud Login page but after entering the login data, it switches to https:// and the error appears.

Again, which IP are you entering? The nextcloud local IP or your public IP?
Your public IP should ALWAYS forward HTTP to HTTPS and not display any webpages via HTTP whatsoever! (HTTPtoHTTPS_rule)

My Network:
ISP --> Modem --> OPNSense --> Proxmox --> Server 1 Nextcloud

Is the firewall in your modem disabled or is it still doing NAT?

Thanks for your input. I changed the cipher accordingly to https://ssl-config.mozilla.org/#server=haproxy&version=2.1&config=modern&openssl=1.1.1d&guideline=5.6 and also applied your rule but still no success.
I am just curios because in the main post the value TLS_AES_128_GCM_SHA256 is also left out 

You won't need any 128 bit ciphers unless you are using very very old devices to access your services which I highly doubt since you only want to enable TLS v1.3 anyway.
So you can safely use the cipher suites in my first post which are identical to the ones from the Mozilla SSL configurator but have the "insecure / weak" 128 bit ciphers removed.

Out of curiosity I tried do enter the only the main domain DOMAIN.dedyn.io and not SUBDOMAIN.DOMAIN.dedyn.io. Then I get a different error

Firefox vertraut dieser Website nicht, weil das von der Website verwendete Zertifikat nicht für DOMAIN.dedyn.io gilt. Das Zertifikat ist nur gültig für *.DOMAIN.dedyn.io.
SSL_ERROR_BAD_CERT_DOMAIN

I can accept the risk but this is something I need to do on all devices...

Could the problem be that I already had DynDNS by another provider and LE certificate? If yes how can I revoke it?

Well, the error is pretty much self-explanatory isn't it?
In my tutorial the wildcard certificate is only valid for the 2nd-level subdomains "*.DOMAIN.dedyn.io" but not for the 1st-level subdomain "DOMAIN.dedyn.io" itself.

If you want the certificate to also cover for "domain.dedyn.io" then you will have to change the certificate in the ACME client to match that. See Part 3 - Step 6.
You will however only need this if you are serving a website in the domain root without "www" infront of it.

Code: [Select]

# currently configured
Common Name = *.DOMAIN.dedyn.io

# what you MIGHT want
Common Name = DOMAIN.dedyn.io
Alt Names = *.DOMAIN.dedyn.io

This will cover the 1st-level subdomain including all 2nd-level subdomains.
Don't forget to reissue the certificate.

129

Tutorials and FAQs / Re: Tutorial 2022/06: HAProxy + Let's Encrypt Wildcard Certificates + 100% A+ Rating

« on: June 11, 2022, 03:20:57 pm »

• 20220611
  
  • Changed the IP address of the "SSL_server" to one that belongs to the localhost subnet.
  • Updated the "NoSSL_condition" based on the HAProxy docs.

130

Tutorials and FAQs / Re: Tutorial 2022/06: HAProxy + Let's Encrypt Wildcard Certificates + 100% A+ Rating

« on: June 10, 2022, 02:14:12 pm »

The issue seems to be the NoSSL_condition. The suggested solution is to replace the condition type from Traffic is SSL (TCP request content inspection)  to Traffic is SSL (locally deciphered):

Code: [Select]

     # logging options
     # ACL: NoSSL_condition
-    acl acl_629f48c6073c95.86527303 req.ssl_ver gt 0
+    acl acl_629f48c6073c95.86527303 ssl_fc
 
     # ACTION: HTTPtoHTTPS_rule
     http-request redirect scheme https code 301 if !acl_629f48c6073c95.86527303

When doing so, the warning is gone. However, this is the first time I'm using HAProxy and I don't really know what I'm doing, so I wanted to check-in with you guys to ensure my solution is correct?

If so, I'd appreciate if you updated the screenshot in step 5.7 @TheHellSite

Thanks for the detailed explanation!
I already had this on my to do list, but then couldn't find the time for it and in the end simply forgot about it.

The HAProxy docs are also stating to use the "ssl_fc" condition.
https://www.haproxy.com/blog/redirect-http-to-https-with-haproxy/

I will change it in the next couple of days!

131

Tutorials and FAQs / Re: Tutorial 2022/06: HAProxy + Let's Encrypt Wildcard Certificates + 100% A+ Rating

« on: June 04, 2022, 03:42:34 pm »

I just updated the tutorial with a very important change to the DynDNS part. It therefore only affects users with a dynamic WAN IP.

• 20220604
  
  • Updated the DynDNS part to use the newer "Dynamic DNS Client" (os-ddclient) plugin.
    The previous "Dynamic DNS Support" (os-dyndns) plugin will be removed with the release of OPNsense 22.7.x.

132

Tutorials and FAQs / Re: Tutorial 2022/04: HAProxy + Let's Encrypt Wildcard Certificates + 100% A+ Rating

« on: April 24, 2022, 11:51:04 am »

I am not sure if this is the correct way to achieve multiple domains pointing to different backends but it seems to be working for me.
At first I ran into a issue were all domains could access the same subdomain, this is when I realized I just needed some extra conditions.

Here are the steps to achieve; service.example.com & service1.example1.com

Services --> ACME Client --> Certificates
Add the certificate for your extra domains and forcefully issue your certificate

Services --> HAProxy --> Settings --> Virtual Services --> Public Services
Finally we edit our "1_HTTPS_frontend"
Add all extra domains in the "Certificates" input.

Just the steps above are necessary and the following step.

Then edit the "PUBLIC_SUBDOMAINS_map" and change the subdomain entries to full FQDNs.

Code: [Select]

old
===
nas NAS_backend

new
===
nas.domain1.com NAS_1_backend
nas.domain2.com NAS_2_backend
Of course you could also create dedicated map-files and rules for each domain, but in terms of functionality it is not necessary.

133

Tutorials and FAQs / Re: Tutorial 2022/04: HAProxy + Let's Encrypt Wildcard Certificates + 100% A+ Rating

« on: April 19, 2022, 08:59:40 pm »

Edit: Ok, changed the Override IP to my LAN IP (10.10.10.1). Now it works.
But to be honest, i do not understand why.

You error explains why!

Code: [Select]

Informational haproxy 10.10.10.206:63264 [19/Apr/2022:17:26:27.483] 1_HTTPS_frontend/10.12.0.1:443: Received something which does not look like a PROXY protocol header
The HTTPS_frontend expects that all data sent to it has the "proxy protocol header".
Since you pointed your internal requests directly to your HTTPS_frontend (HAProxy_VIP) instead of your SNI_frontend (any of the real local IPs of your OPNsense) the data didn't get the PROXY protocol header attached by the SSL_backend.

134

Tutorials and FAQs / Re: Tutorial 2022/04: HAProxy + Let's Encrypt Wildcard Certificates + 100% A+ Rating

« on: April 19, 2022, 06:26:15 pm »

My services are available from the outside.
But from inside they are not accessible.

Well, there you got the point of error.
You probably configured the wrong IP in your DNS overwrites.

What is your OPNsense LAN IP, what is the DNS Overwrite IP and what is the virtual IP of the "HAProxy SSL Server"?

135

Virtual private networks / Re: SiteToSite routing problems with different gateways

« on: April 12, 2022, 07:12:46 am »

Bump.