OPNsense Forum

English Forums => Tutorials and FAQs => Topic started by: boku on April 17, 2025, 08:49:54 AM

Title: HAProxy for IMAP/S help needed
Post by: boku on April 17, 2025, 08:49:54 AM
Hi All

I am struggling with the setup of HAProxy for IMAP/S (for HTTP, I use Caddy). In the HAProxy logs, all I can see is:

Informational   haproxy   Connect from <client IP>:61628 to <OPNsense WAN IP>:993 (public_imaps/TCP)

But my (MacOS Mail) client returns an error. I was told, that the issue is what I can see in the HAProxy config export:

backend pool_dms_imaps
    # health checking is DISABLED
    mode tcp
    balance source
    # stickiness
    stick-table type ip size 50k expire 30m 
    stick on src
    server dms_imaps <internal mail server name>:993 maxconn 100 ssl alpn h2,http/1.1 verify none send-proxy-v2 check-send-proxy

"alpn h2,http/1.1":
HAProxy, as I was told, somehow defaults to HTTP and that is why mail client access fails.

I use:
- Multiplexer Protocol = none in Real Server
- Mode = TCP in Backend Pool
- Type = TCP in Public Service

curl -vk https://<public DNS name>:993
* Host <public DNS name>:993 was resolved.
* IPv6: (none)
* IPv4: <OPNsense WAN IP>
*   Trying <OPNsense WAN IP>:993...
* Connected to <public DNS name> (<OPNsense WAN IP>) port 993
* ALPN: curl offers h2,http/1.1
* (304) (OUT), TLS handshake, Client hello (1):
* (304) (IN), TLS handshake, Server hello (2):
* (304) (IN), TLS handshake, Unknown (8):
* (304) (IN), TLS handshake, Certificate (11):
* (304) (IN), TLS handshake, CERT verify (15):
* (304) (IN), TLS handshake, Finished (20):
* (304) (OUT), TLS handshake, Finished (20):
* SSL connection using TLSv1.3 / AEAD-CHACHA20-POLY1305-SHA256 / [blank] / UNDEF
* ALPN: server accepted h2
* Server certificate:
*  subject: CN=<public DNS name>
*  start date: Mar  3 22:03:37 2025 GMT
*  expire date: Jun  1 22:03:36 2025 GMT
*  issuer: C=US; O=Let's Encrypt; CN=R10
*  SSL certificate verify ok.
* using HTTP/2
* [HTTP/2] [1] OPENED stream for https://<public DNS name>:993/
* [HTTP/2] [1] [:method: GET]
* [HTTP/2] [1] [:scheme: https]
* [HTTP/2] [1] [:authority: <public DNS name>:993]
* [HTTP/2] [1] [:path: /]
* [HTTP/2] [1] [user-agent: curl/8.7.1]
* [HTTP/2] [1] [accept: */*]

To those using HAProxy for IMAP/S. Can you please share what you have configured differently or am I misguided by assuming that "alpn h2,http/1.1" is causing my issue? Can this setup be configured in the UI or only via some config includes via SSH? I couldn't find any hint on that.

Thank you, Patric
Title: Re: HAProxy for IMAP/S help needed
Post by: zerwes on April 17, 2025, 10:01:05 AM
first: I never used IMAPs w/ ALPN, so all below are assumptions. I just know ALPN in combination w/ HTTP(1.2/2)

hm ... I think you mix up things here ... imaps and https are different protocols

mode tcp in your backend cfg. is OK for this.

you should use openssl instead of curl for testing
openssl s_client -connect IPorDNS:993
openssl will log information about ALPN too, so you should first run a check against the original server ant d then against haproxy front
Title: Re: HAProxy for IMAP/S help needed
Post by: Monviech (Cedrik) on April 17, 2025, 10:06:05 AM
Just dropping in here since I read you also use caddy, in it the imaps proxying with tls termination should also work.

Check out the configuration example here, though never tested it myself:

https://github.com/opnsense/plugins/pull/4364

https://docs.opnsense.org/manual/how-tos/caddy.html#caddy-layer4-proxy
Title: Re: HAProxy for IMAP/S help needed
Post by: cookiemonster on April 17, 2025, 10:53:49 AM
we know that by now, all problems with haproxy have a standard answer: run caddy (Trademark Cedrik) ;)

joking aside, yes let's start with openssl s_client indeed.


p.s. Cedrik, really it is a tongue in cheek comment. You are providing an alternative to the problem.
Title: Re: HAProxy for IMAP/S help needed
Post by: Monviech (Cedrik) on April 17, 2025, 12:12:51 PM
I just saw that they already use Caddy in their post so I wanted to point at that it should also support that feature.

I usually leave the only HA proxy threads alone :-)
Title: Re: HAProxy for IMAP/S help needed
Post by: cookiemonster on April 17, 2025, 12:51:38 PM
Quote from: Monviech (Cedrik) on April 17, 2025, 12:12:51 PMI usually leave the only HA proxy threads alone :-)
Please don't on account of a silly comment. I'm sure your inputs would help.
p.s. I wish we could move support questions out of the Tutorials and FAQs section.
Title: Re: HAProxy for IMAP/S help needed
Post by: boku on April 17, 2025, 03:22:26 PM
Quote from: cookiemonster on April 17, 2025, 12:51:38 PM
Quote from: Monviech (Cedrik) on April 17, 2025, 12:12:51 PMI usually leave the only HA proxy threads alone :-)
Please don't on account of a silly comment. I'm sure your inputs would help.
p.s. I wish we could move support questions out of the Tutorials and FAQs section.

I am really sorry for the wrong forum - I didn't find a way to move it to the General questions. Can you please give me a hint how to do that maybe via PM? thanks
Title: Re: HAProxy for IMAP/S help needed
Post by: Monviech (Cedrik) on April 17, 2025, 03:35:16 PM
It's okay, just continue here. :)

No need to micromanage this.
Title: Re: HAProxy for IMAP/S help needed
Post by: boku on April 17, 2025, 08:33:09 PM
Thank you for your feedback and pointing me to a copy/paste mistake which didn't make any sense. HAProxy was a recommendation but if Caddy is capable of the same, I am happy to try Caddy.

These DNS names are relevant:
- mail.beuthen.net: public DNS name pointing to OPNsense (not accessible from the internet at the moment)
- dms.beuthen.net: internal DNS name

I assume, that SSL termination is important so that the mail client can verify the server's certificate CN. As I understand, this is only possible with TLS/SNI, Domain and Terminate SSL = X.

I have tried the Proxy Protocols but then I event got as far as this...

When using TLS:

I seem to reach the Dovecot server which tells me that there are no firewall rule blocking any traffic. I also see matching the FW rule in the logs.

openssl s_client -connect mail.beuthen.net:993
Connecting to 192.168.178.5
CONNECTED(00000005)
depth=2 C=US, O=Internet Security Research Group, CN=ISRG Root X1
verify return:1
depth=1 C=US, O=Let's Encrypt, CN=R10
verify return:1
depth=0 CN=dms.beuthen.net
verify return:1
---
Certificate chain
 0 s:CN=dms.beuthen.net
   i:C=US, O=Let's Encrypt, CN=R10
   a:PKEY: rsaEncryption, 4096 (bit); sigalg: RSA-SHA256
   v:NotBefore: Mar 31 12:01:45 2025 GMT; NotAfter: Jun 29 12:01:44 2025 GMT
 1 s:C=US, O=Let's Encrypt, CN=R10
   i:C=US, O=Internet Security Research Group, CN=ISRG Root X1
   a:PKEY: rsaEncryption, 2048 (bit); sigalg: RSA-SHA256
   v:NotBefore: Mar 13 00:00:00 2024 GMT; NotAfter: Mar 12 23:59:59 2027 GMT
---
Server certificate
-----BEGIN CERTIFICATE-----
MIIGHjCCBQagAwIBAgISBiKgF9BrLRHYuabfcsXsYJO7MA0GCSqGSIb3DQEBCwUA
MDMxCzAJBgNVBAYTAlVTMRYwFAYDVQQKEw1MZXQncyBFbmNyeXB0MQwwCgYDVQQD
EwNSMTAwHhcNMjUwMzMxMTIwMTQ1WhcNMjUwNjI5MTIwMTQ0WjAaMRgwFgYDVQQD
Ew9kbXMuYmV1dGhlbi5uZXQwggIiMA0GCSqGSIb3DQEBAQUAA4ICDwAwggIKAoIC
AQCqukOGW7+POHISZLUT0I0keg0CVRBN59UwRWHcXpJW0gkFMOQ78FG9gIsp9umi
MtOOMz/VBiZ+GtmduxB5n9JH9bND95TazHJxC5DIUI/y0ba2QqS7xmQFhIxxFQT6
X06JDnD9x2Jyrt+5u3GZ12vQsn2tNFaoQUZD3iMWsTNUo4US8Hi1k5q11S8gHXk2
2C1LfVmmiQ7a7IyuXS0vrCJyD7lALLGW6d4fBu/8qiqjRdgt5uGa9c+SZs6tWmSt
HjQaKFzz32ViS/DSHZL9uw+pSEDE1zEI9xBHbOjiULgNQogs3x/p+0gbvX9laptd
YxhyBUSJC1VC8j6z5088+Ph7Z5Ia9u6qUZjA5krukA33QzCeEZlCvu0H6jRhRefc
rf9raXmcn5+pBra+asxYIVR/7a3PhBxOs0h3y/hVlWAilZzWBzG9IioCC4WqFDFz
VS2nKiuXldAkkhzqdeZYzBxTeEFw9AXOj8KydLcsTtIlVnlW+cvHFztKV5LOMC/w
SFGXIUsFnIAnFqeDyh9UE9wTWjiefUQLtF72vaUQmd0QChoRgqbwDTAWezblhO7B
Tl1zZvNMWCzAfvIZlQtfgDtqx1+mSS+BaG5zpRX2+OOg7m3sI2ifMOZRLxCYUH3W
TpIFWYo9ve8DSyLyab+bU3VusCtXUvOPC8JJS3vniBrweQIDAQABo4ICQzCCAj8w
DgYDVR0PAQH/BAQDAgWgMB0GA1UdJQQWMBQGCCsGAQUFBwMBBggrBgEFBQcDAjAM
BgNVHRMBAf8EAjAAMB0GA1UdDgQWBBTfN8MPQx02pKh6PV5i6QndcTkyaTAfBgNV
HSMEGDAWgBS7vMNHpeS8qcbDpHIMEI2iNeHI6DBXBggrBgEFBQcBAQRLMEkwIgYI
KwYBBQUHMAGGFmh0dHA6Ly9yMTAuby5sZW5jci5vcmcwIwYIKwYBBQUHMAKGF2h0
dHA6Ly9yMTAuaS5sZW5jci5vcmcvMBoGA1UdEQQTMBGCD2Rtcy5iZXV0aGVuLm5l
dDATBgNVHSAEDDAKMAgGBmeBDAECATAuBgNVHR8EJzAlMCOgIaAfhh1odHRwOi8v
cjEwLmMubGVuY3Iub3JnLzYwLmNybDCCAQQGCisGAQQB1nkCBAIEgfUEgfIA8AB1
AMz7D2qFcQll/pWbU87psnwi6YVcDZeNtql+VMD+TA2wAAABlexJ0n8AAAQDAEYw
RAIgYhzRHy2dDApDwsfWXrf0Kj3Dbpo8cFiCmyqaW2tcbo0CIEMMKTdV2+7hWNaM
Ok3TB1WPIoHQc/bQT3kZoA4KzW0/AHcAE0rfGrWYQgl4DG/vTHqRpBa3I0nOWFdq
367ap8Kr4CIAAAGV7EnT8gAABAMASDBGAiEAmpbbe0D17QUvZC7VwVtwMcmpjbpq
SYxqdo1Ap+9e6o4CIQDnx38r2kH7ZW6RuGhoxipfWSQxSFGrkJPRW2m8KplDZTAN
BgkqhkiG9w0BAQsFAAOCAQEAGqsLWyXtSma+39EzjGfxeMQuvslXYphCwbSJJn8z
5baUvv5mHhGc8g8x684bfV4ZGzVhfTj3PQFtTjdLzGEOTQWZogCnbcyx1bI90zu0
dVPLBOE03QvZSId3flJESxK7lC94sQ5kzr7+NL8YOAvK+RzyvtNCDlQtVwE4y2hq
UwfvLzPvYFb+/l0oD2AJN0m8jvAG7ss9ziEBL6PkJ8d2m+VPr7V5WfhBMjOPXCRm
RaFxd9sTHdikRNOWZz8n6KAXiQJNQ/gQL4WmDnHwBNibKKl0lyKatSbFNAsa6/Pl
fUh5EpMnkSLNWRIWYs7lHsXlzFCIureLcgVkgkNHP3LQdQ==
-----END CERTIFICATE-----
subject=CN=dms.beuthen.net
issuer=C=US, O=Let's Encrypt, CN=R10
---
No client certificate CA names sent
Peer signing digest: SHA256
Peer signature type: RSA-PSS
Server Temp Key: X25519, 253 bits
---
SSL handshake has read 3680 bytes and written 407 bytes
Verification: OK
---
New, TLSv1.3, Cipher is TLS_AES_256_GCM_SHA384
Protocol: TLSv1.3
Server public key is 4096 bit
This TLS version forbids renegotiation.
Compression: NONE
Expansion: NONE
No ALPN negotiated
Early data was not sent
Verify return code: 0 (ok)
---
---
Post-Handshake New Session Ticket arrived:
SSL-Session:
    Protocol  : TLSv1.3
    Cipher    : TLS_AES_256_GCM_SHA384
    Session-ID: 2F1CD50AEC8B9FA684F562530955EF5022C493B196F26A98A0A69BF70CBA9B38
    Session-ID-ctx:
    Resumption PSK: 157768E8615287E904E71A0DD8D35BE9D15907D3A495B0F0BDD4E91E4FF0F219E763B5CF4CFD5A8495D0A3E5DC8F8286
    PSK identity: None
    PSK identity hint: None
    SRP username: None
    TLS session ticket lifetime hint: 7200 (seconds)
    TLS session ticket:
    0000 - f2 4f 45 7d 69 4e 5b 66-51 e7 8a fc 0c 14 62 c4   .OE}iN[fQ.....b.
    0010 - bb 39 7d 44 9c fe e2 50-58 65 91 58 1b 52 b9 62   .9}D...PXe.X.R.b
    0020 - 1a b2 82 4b e6 26 04 58-72 d9 18 35 96 37 6d 43   ...K.&.Xr..5.7mC
    0030 - f0 0b aa 41 5d 1e 6b 88-29 31 b4 de 19 19 52 e2   ...A].k.)1....R.
    0040 - 41 74 67 0e db 10 68 19-bb 00 02 88 01 f8 0c 16   Atg...h.........
    0050 - 41 30 3e a9 6b 16 24 a7-ac 86 78 e2 7c 5f 5a b9   A0>.k.$...x.|_Z.
    0060 - e7 3f f1 a7 dc 75 6e e0-77 5a 36 f6 5f f7 44 6b   .?...un.wZ6._.Dk
    0070 - 7d 64 50 ba 39 23 99 b8-90 f5 1f 38 35 00 21 f0   }dP.9#.....85.!.
    0080 - a4 be f0 98 11 13 d0 7f-33 22 92 5e ec 02 ff ed   ........3".^....
    0090 - 9f ba 8b a6 5a fd 90 4a-8d 05 a2 77 59 89 3c 6d   ....Z..J...wY.<m
    00a0 - 89 b8 05 d6 6d c1 cc e9-5e c0 f5 06 1c 96 1e 45   ....m...^......E
    00b0 - ee d0 24 1f 4a bc 9c d8-01 64 7f 75 8f 0f ad 33   ..$.J....d.u...3
    00c0 - 52 64 3b 41 3c 40 d8 56-53 4a 2e 2c 6a a6 f7 f3   Rd;A<@.VSJ.,j...
    00d0 - 9c 7d 80 6a 51 46 43 f2-44 02 05 5e 24 e3 fc 3a   .}.jQFC.D..^$..:

    Start Time: 1744908742
    Timeout   : 7200 (sec)
    Verify return code: 0 (ok)
    Extended master secret: no
    Max Early Data: 0
---
read R BLOCK
---
Post-Handshake New Session Ticket arrived:
SSL-Session:
    Protocol  : TLSv1.3
    Cipher    : TLS_AES_256_GCM_SHA384
    Session-ID: 62E55593EC2359EF978297CF85A32F19F4A1B9FD934BBB618788D0A67FC6D906
    Session-ID-ctx:
    Resumption PSK: 6A2F225A452B071D170F75108F172AF8F625B8A8DBAE7ADC9428AEAA707D0E46BAAF799D378725364699E7609B2D6A31
    PSK identity: None
    PSK identity hint: None
    SRP username: None
    TLS session ticket lifetime hint: 7200 (seconds)
    TLS session ticket:
    0000 - f2 4f 45 7d 69 4e 5b 66-51 e7 8a fc 0c 14 62 c4   .OE}iN[fQ.....b.
    0010 - 0d 98 f5 c6 73 7b f0 45-c2 f2 c2 40 0b e2 b8 50   ....s{.E...@...P
    0020 - 60 f3 d7 e1 72 9a 75 91-2d 15 2b 94 64 4d 9b 13   `...r.u.-.+.dM..
    0030 - 6e 59 8f 4b 74 4d ec 67-3c 44 6b 05 67 dc 2f 27   nY.KtM.g<Dk.g./'
    0040 - fb 84 bb 9a c4 53 10 44-8b 06 8a b7 c1 ca 5d d5   .....S.D......].
    0050 - 63 43 10 00 cc d5 66 39-c3 a1 87 7e 75 98 71 29   cC....f9...~u.q)
    0060 - b7 5a fc b8 10 33 1e 86-ce 18 61 8e c7 56 73 7d   .Z...3....a..Vs}
    0070 - 0d 47 ae 42 34 62 c7 2b-8c ff 51 f9 db 84 b2 d6   .G.B4b.+..Q.....
    0080 - bf 4a 78 dd 55 10 3f 10-58 bf a9 4a f1 88 27 fa   .Jx.U.?.X..J..'.
    0090 - d8 15 62 47 9f d0 d3 12-c8 4a cd 5f 38 47 94 70   ..bG.....J._8G.p
    00a0 - 31 08 e5 af f7 60 71 74-05 6d 5a 3f b9 ca 6a cf   1....`qt.mZ?..j.
    00b0 - 97 78 76 6a 29 28 33 3c-e7 da be 2c aa 8c fa 5d   .xvj)(3<...,...]
    00c0 - 18 5a 52 aa 65 46 a6 0e-18 39 1c bd f1 78 c3 39   .ZR.eF...9...x.9
    00d0 - a2 dd 4d cc 5b 7f 91 9e-82 18 3f 6a ff 42 3e 9c   ..M.[.....?j.B>.

    Start Time: 1744908742
    Timeout   : 7200 (sec)
    Verify return code: 0 (ok)
    Extended master secret: no
    Max Early Data: 0
---
read R BLOCK
* OK [CAPABILITY IMAP4rev1 SASL-IR LOGIN-REFERRALS ID ENABLE IDLE LITERAL+ AUTH=PLAIN AUTH=LOGIN AUTH=OAUTHBEARER AUTH=XOAUTH2] Dovecot (Debian) ready.


When using TLS/SNI:

I do not seem to reach the Dovecot server:

openssl s_client -connect mail.beuthen.net:993
Connecting to 192.168.178.5
CONNECTED(00000005)
depth=2 C=US, O=Internet Security Research Group, CN=ISRG Root X1
verify return:1
depth=1 C=US, O=Let's Encrypt, CN=R10
verify return:1
depth=0 CN=mail.beuthen.net
verify return:1
---
Certificate chain
 0 s:CN=mail.beuthen.net
   i:C=US, O=Let's Encrypt, CN=R10
   a:PKEY: rsaEncryption, 4096 (bit); sigalg: RSA-SHA256
   v:NotBefore: Mar  3 22:03:37 2025 GMT; NotAfter: Jun  1 22:03:36 2025 GMT
 1 s:C=US, O=Let's Encrypt, CN=R10
   i:C=US, O=Internet Security Research Group, CN=ISRG Root X1
   a:PKEY: rsaEncryption, 2048 (bit); sigalg: RSA-SHA256
   v:NotBefore: Mar 13 00:00:00 2024 GMT; NotAfter: Mar 12 23:59:59 2027 GMT
---
Server certificate
-----BEGIN CERTIFICATE-----
MIIF8TCCBNmgAwIBAgISBCNHocfJgDIm9zrZnHweWS/PMA0GCSqGSIb3DQEBCwUA
MDMxCzAJBgNVBAYTAlVTMRYwFAYDVQQKEw1MZXQncyBFbmNyeXB0MQwwCgYDVQQD
EwNSMTAwHhcNMjUwMzAzMjIwMzM3WhcNMjUwNjAxMjIwMzM2WjAbMRkwFwYDVQQD
ExBtYWlsLmJldXRoZW4ubmV0MIICIjANBgkqhkiG9w0BAQEFAAOCAg8AMIICCgKC
AgEAsX1iW9BS6JskFMZYl+r3MGVLGWG47ITz6NJ0t01RIcKNOd2ny0PPYfHpvLOO
QiWthejftZfZjvi1s6F/xGuk8BjFMxD/TxGFM+abjxS0IL6vW3fooaZY8DJ8z0xw
LcLK6TIl1GiAIAEPg9yXZHt+eNLl/G2gKNU7FjMjaTIQCFNCXLN1nuX2ERK/1CWw
NOj/UxR6zIM69ON4PU8I+c52PVifkhC9oEbRSMqqgA4p2o0ItgEiYSplqT/9gbLy
m/N/n+1x6Df7rQutGuQhsoX2DhfpK6Qgj0wjs9r3vQ7fDeucp7Os9TZl55GXYVF2
NEER6RtqmI7To87cfSMuN4IvbH8NIY/c8Jo7b7jbSKcRwpTvLGThQsoH9mHro3vA
ttShOp2ppvmRqecpGW/kfmwnfMORWTsrV8tlLeoxdDlLwGO4zjY9hZLE5a4t3pqp
UINuGTMVCMx5O7bujM9CjS6xJLu4/6ZoOSTpTRejcCt7HfQNm5lpU/xjTjRVmPQ8
ghr4KtfY762Pw8qQC48xQ+gWxBXCmMyHbE8NyvU3PdWWYyK/dIuZUGWkcSbQxWC6
YT8xr/RGLNEoOWUo69VI0FrjTng/0hYamuSc641lDjijTsRWmiLb4hu3TpvPBEFs
ZFHQoX9Pkrb7gnuwVMXXhhMuJAKgV4W21VI6BwU7D000b3MCAwEAAaOCAhUwggIR
MA4GA1UdDwEB/wQEAwIFoDAdBgNVHSUEFjAUBggrBgEFBQcDAQYIKwYBBQUHAwIw
DAYDVR0TAQH/BAIwADAdBgNVHQ4EFgQUuRNBJtTf+OmZZOaBSyVGz6C311UwHwYD
VR0jBBgwFoAUu7zDR6XkvKnGw6RyDBCNojXhyOgwVwYIKwYBBQUHAQEESzBJMCIG
CCsGAQUFBzABhhZodHRwOi8vcjEwLm8ubGVuY3Iub3JnMCMGCCsGAQUFBzAChhdo
dHRwOi8vcjEwLmkubGVuY3Iub3JnLzAbBgNVHREEFDASghBtYWlsLmJldXRoZW4u
bmV0MBMGA1UdIAQMMAowCAYGZ4EMAQIBMIIBBQYKKwYBBAHWeQIEAgSB9gSB8wDx
AHcAzPsPaoVxCWX+lZtTzumyfCLphVwNl422qX5UwP5MDbAAAAGVXj7JvgAABAMA
SDBGAiEAvXvb4JbfSwgwoTHmTb6Ejtwob3/usKqOaDHRLHLfeR4CIQDRuMBb36ds
r6Oq8VQ+4n3xVI+FhBvNVgh3G6X4hoIu8QB2AObSMWNAd4zBEEEG13G5zsHSQPaW
hIb7uocyHf0eN45QAAABlV4+ycEAAAQDAEcwRQIhAP+BUfcoR1MYbiBdj4aXpgnD
HhHjE2ZzlPPcOQSoh487AiBhCajocCKDoLfxsr4XF2rRL433lw5DpT9wnCdHNHJl
pDANBgkqhkiG9w0BAQsFAAOCAQEAyqN17RwpMKilEIhbygXuBbY8HT6s/mufH4ux
5ysx9NFw7ZRMj3Vg1jeDoo73CX3mDM0f2PynI+ERLcnoYdECZ/SkhUi6avC2oC4H
ppLpMW5RM6uRb0mUA/NOZxWyNz3UbXR3w9z6c7GUHXd9MQduqVQnnq1QyTjedyor
SqU25u7Xvhg+l+rR7vagW6L6pPflBmJmgQJsq3z7wALuLfrjSwxKGTe5zy4XwDb2
gQ7dElpzp7xaeZjrp5LzIhXfCAWFkGOin73dzALnKnjk+4vni3BFXPEf59zyWbau
cbjrInGv6QhtIuDIJFUMAWnz7q772FQsxAvQkE+A6lUzL+XTXA==
-----END CERTIFICATE-----
subject=CN=mail.beuthen.net
issuer=C=US, O=Let's Encrypt, CN=R10
---
No client certificate CA names sent
Peer signing digest: SHA256
Peer signature type: RSA-PSS
Server Temp Key: X25519, 253 bits
---
SSL handshake has read 3615 bytes and written 391 bytes
Verification: OK
---
New, TLSv1.3, Cipher is TLS_CHACHA20_POLY1305_SHA256
Protocol: TLSv1.3
Server public key is 4096 bit
This TLS version forbids renegotiation.
Compression: NONE
Expansion: NONE
No ALPN negotiated
Early data was not sent
Verify return code: 0 (ok)
---
---
Post-Handshake New Session Ticket arrived:
SSL-Session:
    Protocol  : TLSv1.3
    Cipher    : TLS_CHACHA20_POLY1305_SHA256
    Session-ID: D2774B1B82C27B90D084F848661DC7BDF33C604B6D932D76C7DCD4C796CE46A7
    Session-ID-ctx:
    Resumption PSK: 6FF5B072797551AB2501231C38F34F1B79AED4E7D4AC9A795CB3282ACDE8589D
    PSK identity: None
    PSK identity hint: None
    SRP username: None
    TLS session ticket lifetime hint: 604800 (seconds)
    TLS session ticket:
    0000 - 43 01 a6 73 98 0d c1 9b-f4 98 9d f2 b8 ff 1e 6f   C..s...........o
    0010 - 3c e0 44 7e 7f 13 11 3b-c7 47 eb 4b e6 17 2f 6a   <.D~...;.G.K../j
    0020 - bd ba 8c 71 2c eb 1f 86-d4 2e 15 cf 86 55 63 de   ...q,........Uc.
    0030 - 76 b7 cd ac a6 08 89 59-85 d6 0b 83 af d0 ce b9   v......Y........
    0040 - 24 84 32 fa 9c d7 1d da-c7 67 30 21 ba bc ff 67   $.2......g0!...g
    0050 - 50 d8 1e 0a c0 d6 6c 91-93 94 0f de 60 8b 74 f3   P.....l.....`.t.
    0060 - 06 5a 9f 29 45 46 87 52-a1                        .Z.)EF.R.

    Start Time: 1744908844
    Timeout   : 7200 (sec)
    Verify return code: 0 (ok)
    Extended master secret: no
    Max Early Data: 0
---
read R BLOCK

Is my assumption correct that this is a Caddy misconfiguration?


Caddyfile:
   layer4 {
      import /usr/local/etc/caddy/caddy.d/*.layer4global

      tcp/:993 {
         @451bca01-5c14-4866-8862-32d7285d0419 tls sni mail.beuthen.net

         route @451bca01-5c14-4866-8862-32d7285d0419 {
            tls
            proxy tcp/dms.beuthen.net:993 {
            }
         }
      }
   }


JSON Configuration:
    "layer4": {
      "servers": {
        "srv0": {
          "listen": [
            "tcp/:993"
          ],
          "routes": [
            {
              "handle": [
                {
                  "handler": "tls"
                },
                {
                  "handler": "proxy",
                  "upstreams": [
                    {
                      "dial": [
                        "tcp/dms.beuthen.net:993"
                      ]
                    }
                  ]
                }
              ],
              "match": [
                {
                  "tls": {
                    "sni": [
                      "mail.beuthen.net"
                    ]
                  }
                }
              ]
            }
          ]
        }
      }
    },


Thank you
Title: Re: HAProxy for IMAP/S help needed
Post by: Monviech (Cedrik) on April 17, 2025, 10:09:01 PM
If you want to use tls termination, you need the same domain names configured in reverse proxy, so that certificates can be matched on them. The traffic is then terminated and sent cleartext to the dovecot server, which had to receive it on cleartext imap ports.

If you do /not/ check tls termination, then your dovecot server will receive the traffic encrypted and must offer its own certificate.

I cannot really help here as I have never tested it with mailserver traffic, I only tested standard tls traffic with sni, e.g., https.

If you cannot get it to work or the concepts are not clear, use port forwards or steer back to HA proxy which has a bigger community for these kinda configurations.
Title: Re: HAProxy for IMAP/S help needed
Post by: boku on April 18, 2025, 08:06:32 AM
Hi Cedrik

Just for clarification: With "you need the same domain names configured in reverse proxy" I assume, that the same Domain must be defined in Reverse Proxy with certificate, right? That is the case. What about the ports? Does it only check domain names for certificate lookup or is it also checking for the same port (port 993 in the reverse proxy doesn't make much sense).

When you write "The traffic is then terminated and sent cleartext to the dovecot server". I assume, cleartext doesn't work with IMAP/S, doesn't it automatically SSL-encrypt the connection to Dovecot?

I couldn't find any documentation on the Caddy website. Is this development a feature of the Caddy plugin only?

Apart from setting Caddy Log Level to DEBUG, I couldn't find any way to make Caddy report what is happening.

regards
Title: Re: HAProxy for IMAP/S help needed
Post by: zerwes on April 18, 2025, 08:35:22 AM
Hello boku

In general:

 * with SSL/TLS the cert subject name or a alt name should match the DNS hostname / IP that is used in the client für connecting to the service
   for example in your first openssl s_client session you target mail.beuthen.net and the cert CN is for dms.beuthen.net - here you depend on the negligence of your client while checking the ssl cert: some might work with some special config, other not.

 * I am not familiar with caddy, but in general all these solutions cook with the same water in different pots:
    1. you can route the complete traffic direct to the target server, using the original encryption (e2e). Here the proxy sees only the encrypted packages and is agnostic of the content.
    2. you can terminate the ssl encryption on the proxy. so the client communicates with the proxy encrypted and here the traffic is decrypted. the proxy himself can pass the traffic to the target server unencrypted or can re-encrypt it. But the second step is not transparent to the client, he just sees the proxy.

It seems you want to use the re-encrypted version:
client -> proxy using TLS -> proxy uses SNI in order to decided where to route the traffic and in order to decide what cert to use (this is my understanding of SNI) -> proxy uses TLS to the original server (here the SSL settings must accept the cert from the server! and the proxy must resolve the DNS name of the target server to the right IP [dms.beuthen.net] - /me generally prefers using IPs here, but this is just my taste)

maybe this helps a little bit ... and sniffing the traffic on the proxy might help to ...

regards and good luck
Title: Re: HAProxy for IMAP/S help needed
Post by: boku on April 18, 2025, 09:35:18 AM
Thank you zerwes,

When I change upstream port to 143 and execute:
openssl s_client -connect mail.beuthen.net:993
I finally see this:
read R BLOCK
* OK [CAPABILITY IMAP4rev1 SASL-IR LOGIN-REFERRALS ID ENABLE IDLE LITERAL+ STARTTLS LOGINDISABLED] Dovecot (Debian) ready.
But my MacOS Mail client still cannot connect.

When I switch again to upstream port 993:
openssl s_client -connect mail.beuthen.net:993
I only get:
read R BLOCK

I am not sure if I can execute here:
login "<user>" "<pwd>"
But in DockerMailServer log I see:
dms dovecot: imap-login: Disconnected: Connection closed: SSL_accept() failed: error:0A00010B:SSL routines::wrong version number (no auth attempts in 160 secs): user=<>, rip=10.10.200.1, lip=10.10.200.50, TLS handshaking: SSL_accept() failed: error:0A00010B:SSL routines::wrong version number, session=<ZSX8fAgzfs8KCsgB>

Does that mean the re-encrypted connection between Caddy and Dovecot fails because Dovecot does not accept or cannot handle the request?

Regards
Title: Re: HAProxy for IMAP/S help needed
Post by: boku on April 19, 2025, 05:16:00 PM
Hi All
thank you for your time. Suddenly, HAProxy seems to work fine even with re-encryption although I am 100% sure I had the same settings before.
Have a nice Easter weekend
Title: Re: HAProxy for IMAP/S help needed
Post by: martinwildi on June 07, 2025, 10:51:27 AM
Hi boku
I'm struggeling with the same problem. Could you share your config for IMAPS?
I followed https://forum.opnsense.org/index.php?topic=23339.0 to setup HAProxy for my Webservers which are working fine. But for IMAPS i was not sucessfully.

Thanks & kind regards
Martin
Title: Re: HAProxy for IMAP/S help needed
Post by: meyergru on June 07, 2025, 04:16:46 PM
That linked guide does not cover anything but HTTPS traffic.

With IMAP and SMTP, you will have a hard time to terminate the TLS traffic on HAproxy, because it cannot handle STARTTLS (i.e. opportunistic TLS). The only possible way to to it would be to use implicit TLS on ports 993 and 465. However, many E-Mail clients seem to want STARTTLS.

Also, the specific backend is problematic as well: For example, for Postfix, you will want to have SMTP auth, but usually, this is allowed only, if TLS is active - but on Postfix itself, as it does not know that there is a TLS wrapper active, when you contact it without TLS.

So, this is really a can of worms:

1. You have to make the client work with implicit TLS (i.e. without STARTTLS).
2. You have to make the backend (aka "real server") not offer STARTTLS (which it normally will do, because it thinks there is no TLS layer yet).
3. You have to enable the backend to offer authentication despite no TLS layer being present apparently.
4. Postfix can skip STARTTLS via "smtp_tls_wrappermode=yes", but not selectively per destination.

The easier way is to let Postfix handle TLS termination itself and just use HAproxy as a TCP proxy. which is trivial to set up.
Title: Re: HAProxy for IMAP/S help needed
Post by: meyergru on June 07, 2025, 09:59:18 PM
Here are the settings for TLS termination of SMTP (notice the caveat at the end):

2025-06-07 21_37_38-Settings _ HAProxy _ Services _ OPNsense.mgsoft – Mozilla Firefox.png
2025-06-07 21_40_05-Settings _ HAProxy _ Services _ OPNsense.mgsoft – Mozilla Firefox.png
2025-06-07 21_41_31-Settings _ HAProxy _ Services _ OPNsense.mgsoft – Mozilla Firefox.png

In Postfix, you will have to use something like this in master.cf:

# Backend to HAproxy: no TLS, but use authentication anyway
10465       inet  n       -       y       -       -       smtpd
  -o smtpd_tls_security_level=none
  -o smtpd_tls_auth_only=no
  -o smtpd_upstream_proxy_protocol=haproxy

This is to use no TLS for the backend, but allow for authentication anyway. Also, you can specify that HAproxy is used as an upstream proxy. This is mirrored by the "Proxy Protocol" setting in the backend pool. Also, there is a "send-proxy-v2-ssl" option in the real server settings in order to pass SSL options to Postfix.

However, note, that becaus HAproxy cannot handle STARTTLS, this connections always expects a TLS handshake at start (this is the old SUBMISSIONS/SMTPS protocol). That means, that e.g. a Postfix SMTP client must use smtp_tls_wrappermode=yes to skip STARTTLS. This option is global, though, so you cannot easily specify that for specific destinations.
Title: Re: HAProxy for IMAP/S help needed
Post by: meyergru on June 07, 2025, 10:05:32 PM
And these are the settings for TLS termination of IMAP:

2025-06-07 21_38_52-Settings _ HAProxy _ Services _ OPNsense.mgsoft – Mozilla Firefox.png
2025-06-07 21_40_32-Settings _ HAProxy _ Services _ OPNsense.mgsoft – Mozilla Firefox.png
2025-06-07 21_45_07-Settings _ HAProxy _ Services _ OPNsense.mgsoft – Mozilla Firefox.png

In Dovecot, you will need to set something like:

haproxy_trusted_networks = 192.168.1.1

service imap-login {
  inet_listener imapha {
    port = 10143
    haproxy = yes
  }
}

This is to enable the HAproxy protocol and to allow it to be used from a specific IP (or range).

As with the SMTP backend, you will need to enable the V2 proxy protocol, including the SSL options.
Title: Re: HAProxy for IMAP/S help needed
Post by: martinwildi on June 26, 2025, 06:36:23 PM
Hey meyergru

Thanks a lot for this excellent walkthrough!

Finally i found some time to read it carefully and try to understand it ;).

I was aware that the linked guide does only cover HTTPS traffic. Until now i only needed HTTPS. Also i could successfully publish the mailcow UI over HTTPS.

Where i'm not sure if i understand that correctly: the documentation you provided is to terminate TLS Traffic for IMAP and SMTP on HAProxy?

This is not my primary goal. For me it would be perfect if Postfix handles the TLS termination itself and i could just use HAproxy as a TCP proxy. But would i then have to have a Certificate (normally Lets encrypt) on the mailcow server, correct?

I read about solutions to let OPNSense create the LE-certificates and copy it to mailcow. But i want to avoid this if possible.

My goal would be:
HTTPS: as already working: termination on HAProxy, Certificate on HAProxy/OPNSense
IMAPS: proxing to mailcow but with certificate-handling on HAProxy/OPNSense
SMTP (incoming and outgoing): direct

Maybe you have a solutions for this also :)

Thanks again, i really appreciate your help!

regards
Martin
Title: Re: HAProxy for IMAP/S help needed
Post by: meyergru on June 27, 2025, 09:00:39 AM
That should work the way I showed. However, HAproxy can only do explicit TLS, no STARTTLS. Thus, you cannot do a Standard SMTP server on port 25 that can do TLS upgrade.