Messages

1

Tutorials and FAQs / Re: Tutorial 2023/03: HAProxy + Let's Encrypt Wildcard Certificates + 100% A+ Rating

« on: March 21, 2023, 11:07:57 am »

With a "wrong" MTU, shouldn't the VPN connection be shaky with every device from every "outside" network? It's working 100% all the time on my mobile internet (and my girlfriend's as well), and it's working 100% all the time for ssllabs but only some "Is it down or just me"-kinda sites. But for at least 2 friends (one using the newest Opera Browser on Windows), there are timeouts while trying to connect to my websites. Same for a Windows VPS hosted on AWS - can't get a handshake there either (using newest Chrome browser).

I got no problems with my mailserver/proxmox mail gateway at home. I got no hickups with SSH via NAT. I got no issues with gaming servers at home (friends can connect to it). So I think, something isn't working correctly with my haproxy, sadly

That depends on PMTU discovery, so not every connection has to fail. Try reducing MTU/MSS just for the wireguard interface group like so:

OMG, this fixed it! Now I can reach my addresses even with before problematic peers. Thank you so much @thehellsite and @meyergru!

Root Domains
Now I got another question. Did I understand it right, that the tutorial is only working with subdomains, not with root domains? I think, I would have to setup rules to achieve redirects from example.com to www.example.com right?

2

Tutorials and FAQs / Re: Tutorial 2023/03: HAProxy + Let's Encrypt Wildcard Certificates + 100% A+ Rating

« on: March 10, 2023, 10:00:11 am »

Good Morning HellSite!

thank you for your very detailed reply.

I was using your cipher list and cipher suits, before I tried everything to fix the issue. I got the list and suits from https://ssl-config.mozilla.org/ and used "intermediate" because it made somehow sense to me.

I now switched it back to your list and suits and this is the outcome:


I've got a FibreChannel Connection from Deutsche Glasfaser at home (therefore I need the VPS OPNsense to get a static public IP at home). I didn't change the MTU for Wireguard because the mentioned guide used values for VDSL. So I'm running on default 1500 MTU.

With a "wrong" MTU, shouldn't the VPN connection be shaky with every device from every "outside" network? It's working 100% all the time on my mobile internet (and my girlfriend's as well), and it's working 100% all the time for ssllabs but only some "Is it down or just me"-kinda sites. But for at least 2 friends (one using the newest Opera Browser on Windows), there are timeouts while trying to connect to my websites. Same for a Windows VPS hosted on AWS - can't get a handshake there either (using newest Chrome browser).

I got no problems with my mailserver/proxmox mail gateway at home. I got no hickups with SSH via NAT. I got no issues with gaming servers at home (friends can connect to it). So I think, something isn't working correctly with my haproxy, sadly

Edit: Home MTU is at the Wireguard default value = 1420. In the VPS it's set to 1340 (nowhere changed though - so it must have decided to use this MTU automatically on the Wireguard interface). And the VPS Wireguard Interace has some error (in/out - 1 / 688).

Changed the VPS MTU to 1420 as well. Errors now 5/3 since tunnel restart. But nothing else changed. I still get "ERR_TIMED_OUT" on my websites from AWS VPS.

3

Tutorials and FAQs / Re: Tutorial 2023/03: HAProxy + Let's Encrypt Wildcard Certificates + 100% A+ Rating

« on: March 09, 2023, 10:14:58 am »

Hey everyone! Hey TheHellSite!

First of all, I'm really thankful for your effort to write this really good guide and give support to date! That's awesome and should be honoured.

A bit of a backstory before I dive into the problem.
I'm a system engineer working with Sophos Products for years at enterprises. Because of that, I was using Sophos UTM Home Editions for years for my own private  servers as well -  but wanted to make the switch to OPNsense long ago. Some months ago, I finally decided to work on the switch. Your guide supported me so hard to make this switch from Sophos UTM to OPNsense.

Fast forward
I was thinking, my haproxy on my OPNsense was working completely. I had some issues before, where I could render websites from my local network (altough not using Split DNS or similar - just public IPs), but not from the internet (tested this with my 5G connection from my phone). After some tinkering, I got it even working on my phone with 5G and thought, everything should be okay now.

My Setup
I'm using 2 OPNsense, one at home and one in a datacenter on a VPS. I have these 2 connected to eachother via a Wireguard Tunnel (OPNsense plugin) using this guide (sorry, it's German: https://www.busche.org/index.php/2021/03/21/ipv4-ueber-wireguard-von-opnsense-zu-opnsense-routen-cgnat-umgehen/). And I'm using exactly your guide with the only difference using my wireguard interface instead of WAN for firewallrules.

Wireguard is working awesome, leads all traffic via Proxy ARP virtual IP (2nd public IP on VPS) on the VPS OPNsense to my home OPNsense.

Now the problem
More often then not, it seems that my websites aren't reachable from the internet. It's working from my 5G Internet Connection, it's working for a webdeveloper, who was assisting me with a web-project from hungary. But when friends test it or when I test it from my AWS Windows Machine, it's running into a timeout. Same for multiple status checking sites like https://isitdownorjust.me/. https://www.ssllabs.com/ssltest/ on the other hand is able to reach and check my websites.

I checked the haproxy logs and in those cases where it doesn't work, I get [09/Mar/2023:09:50:14.471] 1_HTTPS_frontend/127.0.0.1:443: SSL handshake failure

I checked your tutorial like 100 times. Checked my config side by side. It doesn't make any sense in my mind, that it works in some cases, and in some other cases it doesn't. And it drives me crazy. I was trying to fix this issue myself for weeks before I decided to write a comment and ask for help here.

My Config Export looks like this (deleted most backends and only left 2 for reference):

Code: [Select]

#
# Automatically generated configuration.
# Do not edit this file manually.
#

global
    uid                         80
    gid                         80
    chroot                      /var/haproxy
    daemon
    stats                       socket /var/run/haproxy.socket group proxy mode 775 level admin
    nbthread                    4
    hard-stop-after             60s
    no strict-limits
    maxconn                     10000
    tune.ssl.default-dh-param   4096
    spread-checks               2
    tune.bufsize                16384
    tune.lua.maxmem             0
    log                         /var/run/log local0 info
    lua-prepend-path            /tmp/haproxy/lua/?.lua

defaults
    log     global
    option redispatch -1
    maxconn 5000
    timeout client 30s
    timeout connect 30s
    timeout server 30s
    retries 3
    default-server init-addr last,libc

# autogenerated entries for ACLs


# autogenerated entries for config in backends/frontends


# autogenerated entries for stats



# Frontend: 0_SNI_frontend (Listening on 0.0.0.0:80, 0.0.0.0:443)
frontend 0_SNI_frontend
    bind 0.0.0.0:443 name 0.0.0.0:443
    bind 0.0.0.0:80 name 0.0.0.0:80
    mode tcp
    default_backend SSL_backend

    # logging options

# Frontend: 1_HTTP_frontend (Listening on 127.0.0.1:80)
frontend 1_HTTP_frontend
    bind 127.0.0.1:80 name 127.0.0.1:80 accept-proxy
    mode http
    option http-keep-alive
    option forwardfor

    # logging options
    # ACL: NoSSL_Condition
    acl acl_6293a5ef2e36e8.09400894 ssl_fc

    # ACTION: HTTPtoHTTPS_rule
    http-request redirect scheme https code 301 if !acl_6293a5ef2e36e8.09400894

# Frontend: 1_HTTPS_frontend (Listening on 127.0.0.1:443)
frontend 1_HTTPS_frontend
    http-response set-header Strict-Transport-Security "max-age=63072000; includeSubDomains; preload"
    bind 127.0.0.1:443 name 127.0.0.1:443 accept-proxy ssl curves secp384r1  no-sslv3 no-tlsv10 no-tlsv11 no-tls-tickets prefer-client-ciphers ciphers ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-CHACHA20-POLY1305:ECDHE-RSA-CHACHA20-POLY1305:DHE-RSA-AES128-GCM-SHA256:DHE-RSA-AES256-GCM-SHA384 ciphersuites TLS_AES_128_GCM_SHA256:TLS_AES_256_GCM_SHA384:TLS_CHACHA20_POLY1305_SHA256 alpn h2,http/1.1 crt-list /tmp/haproxy/ssl/6293aa32edc294.46241266.certlist
    mode http
    option http-keep-alive
    option forwardfor
    timeout client 15m

    # logging options
    option httplog

    # ACTION: PUBLIC_SUBDOMAINS_map-rule
    # NOTE: actions with no ACLs/conditions will always match
    use_backend %[req.hdr(host),lower,map_dom(/tmp/haproxy/mapfiles/6293a65a1027d4.72742608.txt)]
    # WARNING: pass through options below this line
    # add X-Forwarded-Proto
    http-request set-header X-Forwarded-Proto https if { ssl_fc }

# Backend: SSL_backend ()
backend SSL_backend
    # health checking is DISABLED
    mode tcp
    balance source
    # stickiness
    stick-table type ip size 50k expire 30m 
    stick on src
    server SSL_server 127.0.0.1 send-proxy-v2 check-send-proxy

# Backend: nextcloud-mf_backend ()
backend nextcloud-mf_backend
    # health checking is DISABLED
    mode http
    balance source
    # stickiness
    stick-table type ip size 50k expire 30m 
    stick on src
    http-reuse safe
    server nextcloud-mf nextcloud-mf.fuchsbau.local:443 ssl verify none

# Backend: survey_backend ()
backend survey_backend
    # health checking is DISABLED
    mode http
    balance source
    # stickiness
    stick-table type ip size 50k expire 30m 
    stick on src
    http-reuse safe
    server adv-survey 10.0.110.5:80

listen local_statistics
    bind            127.0.0.1:8822
    mode            http
    stats uri       /haproxy?stats
    stats realm     HAProxy\ statistics
    stats admin     if TRUE

# remote statistics are DISABLED


I hope someone can help me, because I don't want to use the Sophos UTM as a fallback anymore and I'm feeling like I'm losing my mind.

Thanks in advance!