Messages

When I initially installed OPNsense, I read that it is NOT recommended to use ZFS in a Proxmox VM because ZFS is already used for the Proxmox host.

I beg to differ: https://forum.opnsense.org/index.php?topic=44159.0

Fantastic HowTo - thank you very much, I will wipe it out and install it again

Hello all

I assume all this discussion about N100 and UFS filesystem is a general recommendation for bare metal installations as well as for Proxmox VM?

As proposed in this thread, I newly installed OPNsense 25.1 in a new Proxmox VM with filesystem ZFS and successfully upgraded to 25.7. Thank you for sharing your experiences here!

When I initially installed OPNsense, I read that it is NOT recommended to use ZFS in a Proxmox VM because ZFS is already used for the Proxmox host.

regards

Hello,
could you please share what those "3 tunables" are and what you changed? I seem to be missing something. I also use N100 with ZFS any my attempts to upgrade failed 1 week ago - since I haven't heard of single user mode before and rolled back, I simply don't know.

Hi All
thank you for your time. Suddenly, HAProxy seems to work fine even with re-encryption although I am 100% sure I had the same settings before.
Have a nice Easter weekend

Thank you zerwes,

When I change upstream port to 143 and execute:
openssl s_client -connect mail.beuthen.net:993
I finally see this:
read R BLOCK
* OK [CAPABILITY IMAP4rev1 SASL-IR LOGIN-REFERRALS ID ENABLE IDLE LITERAL+ STARTTLS LOGINDISABLED] Dovecot (Debian) ready.
But my MacOS Mail client still cannot connect.

When I switch again to upstream port 993:
openssl s_client -connect mail.beuthen.net:993
I only get:
read R BLOCK

I am not sure if I can execute here:
login "<user>" "<pwd>"
But in DockerMailServer log I see:
dms dovecot: imap-login: Disconnected: Connection closed: SSL_accept() failed: error:0A00010B:SSL routines::wrong version number (no auth attempts in 160 secs): user=<>, rip=10.10.200.1, lip=10.10.200.50, TLS handshaking: SSL_accept() failed: error:0A00010B:SSL routines::wrong version number, session=<ZSX8fAgzfs8KCsgB>

Does that mean the re-encrypted connection between Caddy and Dovecot fails because Dovecot does not accept or cannot handle the request?

Regards

Hi Cedrik

Just for clarification: With "you need the same domain names configured in reverse proxy" I assume, that the same Domain must be defined in Reverse Proxy with certificate, right? That is the case. What about the ports? Does it only check domain names for certificate lookup or is it also checking for the same port (port 993 in the reverse proxy doesn't make much sense).

When you write "The traffic is then terminated and sent cleartext to the dovecot server". I assume, cleartext doesn't work with IMAP/S, doesn't it automatically SSL-encrypt the connection to Dovecot?

I couldn't find any documentation on the Caddy website. Is this development a feature of the Caddy plugin only?

Apart from setting Caddy Log Level to DEBUG, I couldn't find any way to make Caddy report what is happening.

regards

Thank you for your feedback and pointing me to a copy/paste mistake which didn't make any sense. HAProxy was a recommendation but if Caddy is capable of the same, I am happy to try Caddy.

These DNS names are relevant:
- mail.beuthen.net: public DNS name pointing to OPNsense (not accessible from the internet at the moment)
- dms.beuthen.net: internal DNS name

I assume, that SSL termination is important so that the mail client can verify the server's certificate CN. As I understand, this is only possible with TLS/SNI, Domain and Terminate SSL = X.

I have tried the Proxy Protocols but then I event got as far as this...

When using TLS:

I seem to reach the Dovecot server which tells me that there are no firewall rule blocking any traffic. I also see matching the FW rule in the logs.

openssl s_client -connect mail.beuthen.net:993
Connecting to 192.168.178.5
CONNECTED(00000005)
depth=2 C=US, O=Internet Security Research Group, CN=ISRG Root X1
verify return:1
depth=1 C=US, O=Let's Encrypt, CN=R10
verify return:1
depth=0 CN=dms.beuthen.net
verify return:1
---
Certificate chain
 0 s:CN=dms.beuthen.net
   i:C=US, O=Let's Encrypt, CN=R10
   a:PKEY: rsaEncryption, 4096 (bit); sigalg: RSA-SHA256
   v:NotBefore: Mar 31 12:01:45 2025 GMT; NotAfter: Jun 29 12:01:44 2025 GMT
 1 s:C=US, O=Let's Encrypt, CN=R10
   i:C=US, O=Internet Security Research Group, CN=ISRG Root X1
   a:PKEY: rsaEncryption, 2048 (bit); sigalg: RSA-SHA256
   v:NotBefore: Mar 13 00:00:00 2024 GMT; NotAfter: Mar 12 23:59:59 2027 GMT
---
Server certificate
-----BEGIN CERTIFICATE-----
MIIGHjCCBQagAwIBAgISBiKgF9BrLRHYuabfcsXsYJO7MA0GCSqGSIb3DQEBCwUA
MDMxCzAJBgNVBAYTAlVTMRYwFAYDVQQKEw1MZXQncyBFbmNyeXB0MQwwCgYDVQQD
EwNSMTAwHhcNMjUwMzMxMTIwMTQ1WhcNMjUwNjI5MTIwMTQ0WjAaMRgwFgYDVQQD
Ew9kbXMuYmV1dGhlbi5uZXQwggIiMA0GCSqGSIb3DQEBAQUAA4ICDwAwggIKAoIC
AQCqukOGW7+POHISZLUT0I0keg0CVRBN59UwRWHcXpJW0gkFMOQ78FG9gIsp9umi
MtOOMz/VBiZ+GtmduxB5n9JH9bND95TazHJxC5DIUI/y0ba2QqS7xmQFhIxxFQT6
X06JDnD9x2Jyrt+5u3GZ12vQsn2tNFaoQUZD3iMWsTNUo4US8Hi1k5q11S8gHXk2
2C1LfVmmiQ7a7IyuXS0vrCJyD7lALLGW6d4fBu/8qiqjRdgt5uGa9c+SZs6tWmSt
HjQaKFzz32ViS/DSHZL9uw+pSEDE1zEI9xBHbOjiULgNQogs3x/p+0gbvX9laptd
YxhyBUSJC1VC8j6z5088+Ph7Z5Ia9u6qUZjA5krukA33QzCeEZlCvu0H6jRhRefc
rf9raXmcn5+pBra+asxYIVR/7a3PhBxOs0h3y/hVlWAilZzWBzG9IioCC4WqFDFz
VS2nKiuXldAkkhzqdeZYzBxTeEFw9AXOj8KydLcsTtIlVnlW+cvHFztKV5LOMC/w
SFGXIUsFnIAnFqeDyh9UE9wTWjiefUQLtF72vaUQmd0QChoRgqbwDTAWezblhO7B
Tl1zZvNMWCzAfvIZlQtfgDtqx1+mSS+BaG5zpRX2+OOg7m3sI2ifMOZRLxCYUH3W
TpIFWYo9ve8DSyLyab+bU3VusCtXUvOPC8JJS3vniBrweQIDAQABo4ICQzCCAj8w
DgYDVR0PAQH/BAQDAgWgMB0GA1UdJQQWMBQGCCsGAQUFBwMBBggrBgEFBQcDAjAM
BgNVHRMBAf8EAjAAMB0GA1UdDgQWBBTfN8MPQx02pKh6PV5i6QndcTkyaTAfBgNV
HSMEGDAWgBS7vMNHpeS8qcbDpHIMEI2iNeHI6DBXBggrBgEFBQcBAQRLMEkwIgYI
KwYBBQUHMAGGFmh0dHA6Ly9yMTAuby5sZW5jci5vcmcwIwYIKwYBBQUHMAKGF2h0
dHA6Ly9yMTAuaS5sZW5jci5vcmcvMBoGA1UdEQQTMBGCD2Rtcy5iZXV0aGVuLm5l
dDATBgNVHSAEDDAKMAgGBmeBDAECATAuBgNVHR8EJzAlMCOgIaAfhh1odHRwOi8v
cjEwLmMubGVuY3Iub3JnLzYwLmNybDCCAQQGCisGAQQB1nkCBAIEgfUEgfIA8AB1
AMz7D2qFcQll/pWbU87psnwi6YVcDZeNtql+VMD+TA2wAAABlexJ0n8AAAQDAEYw
RAIgYhzRHy2dDApDwsfWXrf0Kj3Dbpo8cFiCmyqaW2tcbo0CIEMMKTdV2+7hWNaM
Ok3TB1WPIoHQc/bQT3kZoA4KzW0/AHcAE0rfGrWYQgl4DG/vTHqRpBa3I0nOWFdq
367ap8Kr4CIAAAGV7EnT8gAABAMASDBGAiEAmpbbe0D17QUvZC7VwVtwMcmpjbpq
SYxqdo1Ap+9e6o4CIQDnx38r2kH7ZW6RuGhoxipfWSQxSFGrkJPRW2m8KplDZTAN
BgkqhkiG9w0BAQsFAAOCAQEAGqsLWyXtSma+39EzjGfxeMQuvslXYphCwbSJJn8z
5baUvv5mHhGc8g8x684bfV4ZGzVhfTj3PQFtTjdLzGEOTQWZogCnbcyx1bI90zu0
dVPLBOE03QvZSId3flJESxK7lC94sQ5kzr7+NL8YOAvK+RzyvtNCDlQtVwE4y2hq
UwfvLzPvYFb+/l0oD2AJN0m8jvAG7ss9ziEBL6PkJ8d2m+VPr7V5WfhBMjOPXCRm
RaFxd9sTHdikRNOWZz8n6KAXiQJNQ/gQL4WmDnHwBNibKKl0lyKatSbFNAsa6/Pl
fUh5EpMnkSLNWRIWYs7lHsXlzFCIureLcgVkgkNHP3LQdQ==
-----END CERTIFICATE-----
subject=CN=dms.beuthen.net
issuer=C=US, O=Let's Encrypt, CN=R10
---
No client certificate CA names sent
Peer signing digest: SHA256
Peer signature type: RSA-PSS
Server Temp Key: X25519, 253 bits
---
SSL handshake has read 3680 bytes and written 407 bytes
Verification: OK
---
New, TLSv1.3, Cipher is TLS_AES_256_GCM_SHA384
Protocol: TLSv1.3
Server public key is 4096 bit
This TLS version forbids renegotiation.
Compression: NONE
Expansion: NONE
No ALPN negotiated
Early data was not sent
Verify return code: 0 (ok)
---
---
Post-Handshake New Session Ticket arrived:
SSL-Session:
    Protocol  : TLSv1.3
    Cipher    : TLS_AES_256_GCM_SHA384
    Session-ID: 2F1CD50AEC8B9FA684F562530955EF5022C493B196F26A98A0A69BF70CBA9B38
    Session-ID-ctx:
    Resumption PSK: 157768E8615287E904E71A0DD8D35BE9D15907D3A495B0F0BDD4E91E4FF0F219E763B5CF4CFD5A8495D0A3E5DC8F8286
    PSK identity: None
    PSK identity hint: None
    SRP username: None
    TLS session ticket lifetime hint: 7200 (seconds)
    TLS session ticket:
    0000 - f2 4f 45 7d 69 4e 5b 66-51 e7 8a fc 0c 14 62 c4   .OE}iN[fQ.....b.
    0010 - bb 39 7d 44 9c fe e2 50-58 65 91 58 1b 52 b9 62   .9}D...PXe.X.R.b
    0020 - 1a b2 82 4b e6 26 04 58-72 d9 18 35 96 37 6d 43   ...K.&.Xr..5.7mC
    0030 - f0 0b aa 41 5d 1e 6b 88-29 31 b4 de 19 19 52 e2   ...A].k.)1....R.
    0040 - 41 74 67 0e db 10 68 19-bb 00 02 88 01 f8 0c 16   Atg...h.........
    0050 - 41 30 3e a9 6b 16 24 a7-ac 86 78 e2 7c 5f 5a b9   A0>.k.$...x.|_Z.
    0060 - e7 3f f1 a7 dc 75 6e e0-77 5a 36 f6 5f f7 44 6b   .?...un.wZ6._.Dk
    0070 - 7d 64 50 ba 39 23 99 b8-90 f5 1f 38 35 00 21 f0   }dP.9#.....85.!.
    0080 - a4 be f0 98 11 13 d0 7f-33 22 92 5e ec 02 ff ed   ........3".^....
    0090 - 9f ba 8b a6 5a fd 90 4a-8d 05 a2 77 59 89 3c 6d   ....Z..J...wY.<m
    00a0 - 89 b8 05 d6 6d c1 cc e9-5e c0 f5 06 1c 96 1e 45   ....m...^......E
    00b0 - ee d0 24 1f 4a bc 9c d8-01 64 7f 75 8f 0f ad 33   ..$.J....d.u...3
    00c0 - 52 64 3b 41 3c 40 d8 56-53 4a 2e 2c 6a a6 f7 f3   Rd;A<@.VSJ.,j...
    00d0 - 9c 7d 80 6a 51 46 43 f2-44 02 05 5e 24 e3 fc 3a   .}.jQFC.D..^$..:

    Start Time: 1744908742
    Timeout   : 7200 (sec)
    Verify return code: 0 (ok)
    Extended master secret: no
    Max Early Data: 0
---
read R BLOCK
---
Post-Handshake New Session Ticket arrived:
SSL-Session:
    Protocol  : TLSv1.3
    Cipher    : TLS_AES_256_GCM_SHA384
    Session-ID: 62E55593EC2359EF978297CF85A32F19F4A1B9FD934BBB618788D0A67FC6D906
    Session-ID-ctx:
    Resumption PSK: 6A2F225A452B071D170F75108F172AF8F625B8A8DBAE7ADC9428AEAA707D0E46BAAF799D378725364699E7609B2D6A31
    PSK identity: None
    PSK identity hint: None
    SRP username: None
    TLS session ticket lifetime hint: 7200 (seconds)
    TLS session ticket:
    0000 - f2 4f 45 7d 69 4e 5b 66-51 e7 8a fc 0c 14 62 c4   .OE}iN[fQ.....b.
    0010 - 0d 98 f5 c6 73 7b f0 45-c2 f2 c2 40 0b e2 b8 50   ....s{.E...@...P
    0020 - 60 f3 d7 e1 72 9a 75 91-2d 15 2b 94 64 4d 9b 13   `...r.u.-.+.dM..
    0030 - 6e 59 8f 4b 74 4d ec 67-3c 44 6b 05 67 dc 2f 27   nY.KtM.g<Dk.g./'
    0040 - fb 84 bb 9a c4 53 10 44-8b 06 8a b7 c1 ca 5d d5   .....S.D......].
    0050 - 63 43 10 00 cc d5 66 39-c3 a1 87 7e 75 98 71 29   cC....f9...~u.q)
    0060 - b7 5a fc b8 10 33 1e 86-ce 18 61 8e c7 56 73 7d   .Z...3....a..Vs}
    0070 - 0d 47 ae 42 34 62 c7 2b-8c ff 51 f9 db 84 b2 d6   .G.B4b.+..Q.....
    0080 - bf 4a 78 dd 55 10 3f 10-58 bf a9 4a f1 88 27 fa   .Jx.U.?.X..J..'.
    0090 - d8 15 62 47 9f d0 d3 12-c8 4a cd 5f 38 47 94 70   ..bG.....J._8G.p
    00a0 - 31 08 e5 af f7 60 71 74-05 6d 5a 3f b9 ca 6a cf   1....`qt.mZ?..j.
    00b0 - 97 78 76 6a 29 28 33 3c-e7 da be 2c aa 8c fa 5d   .xvj)(3<...,...]
    00c0 - 18 5a 52 aa 65 46 a6 0e-18 39 1c bd f1 78 c3 39   .ZR.eF...9...x.9
    00d0 - a2 dd 4d cc 5b 7f 91 9e-82 18 3f 6a ff 42 3e 9c   ..M.[.....?j.B>.

    Start Time: 1744908742
    Timeout   : 7200 (sec)
    Verify return code: 0 (ok)
    Extended master secret: no
    Max Early Data: 0
---
read R BLOCK
* OK [CAPABILITY IMAP4rev1 SASL-IR LOGIN-REFERRALS ID ENABLE IDLE LITERAL+ AUTH=PLAIN AUTH=LOGIN AUTH=OAUTHBEARER AUTH=XOAUTH2] Dovecot (Debian) ready.


When using TLS/SNI:

I do not seem to reach the Dovecot server:

openssl s_client -connect mail.beuthen.net:993
Connecting to 192.168.178.5
CONNECTED(00000005)
depth=2 C=US, O=Internet Security Research Group, CN=ISRG Root X1
verify return:1
depth=1 C=US, O=Let's Encrypt, CN=R10
verify return:1
depth=0 CN=mail.beuthen.net
verify return:1
---
Certificate chain
 0 s:CN=mail.beuthen.net
   i:C=US, O=Let's Encrypt, CN=R10
   a:PKEY: rsaEncryption, 4096 (bit); sigalg: RSA-SHA256
   v:NotBefore: Mar  3 22:03:37 2025 GMT; NotAfter: Jun  1 22:03:36 2025 GMT
 1 s:C=US, O=Let's Encrypt, CN=R10
   i:C=US, O=Internet Security Research Group, CN=ISRG Root X1
   a:PKEY: rsaEncryption, 2048 (bit); sigalg: RSA-SHA256
   v:NotBefore: Mar 13 00:00:00 2024 GMT; NotAfter: Mar 12 23:59:59 2027 GMT
---
Server certificate
-----BEGIN CERTIFICATE-----
MIIF8TCCBNmgAwIBAgISBCNHocfJgDIm9zrZnHweWS/PMA0GCSqGSIb3DQEBCwUA
MDMxCzAJBgNVBAYTAlVTMRYwFAYDVQQKEw1MZXQncyBFbmNyeXB0MQwwCgYDVQQD
EwNSMTAwHhcNMjUwMzAzMjIwMzM3WhcNMjUwNjAxMjIwMzM2WjAbMRkwFwYDVQQD
ExBtYWlsLmJldXRoZW4ubmV0MIICIjANBgkqhkiG9w0BAQEFAAOCAg8AMIICCgKC
AgEAsX1iW9BS6JskFMZYl+r3MGVLGWG47ITz6NJ0t01RIcKNOd2ny0PPYfHpvLOO
QiWthejftZfZjvi1s6F/xGuk8BjFMxD/TxGFM+abjxS0IL6vW3fooaZY8DJ8z0xw
LcLK6TIl1GiAIAEPg9yXZHt+eNLl/G2gKNU7FjMjaTIQCFNCXLN1nuX2ERK/1CWw
NOj/UxR6zIM69ON4PU8I+c52PVifkhC9oEbRSMqqgA4p2o0ItgEiYSplqT/9gbLy
m/N/n+1x6Df7rQutGuQhsoX2DhfpK6Qgj0wjs9r3vQ7fDeucp7Os9TZl55GXYVF2
NEER6RtqmI7To87cfSMuN4IvbH8NIY/c8Jo7b7jbSKcRwpTvLGThQsoH9mHro3vA
ttShOp2ppvmRqecpGW/kfmwnfMORWTsrV8tlLeoxdDlLwGO4zjY9hZLE5a4t3pqp
UINuGTMVCMx5O7bujM9CjS6xJLu4/6ZoOSTpTRejcCt7HfQNm5lpU/xjTjRVmPQ8
ghr4KtfY762Pw8qQC48xQ+gWxBXCmMyHbE8NyvU3PdWWYyK/dIuZUGWkcSbQxWC6
YT8xr/RGLNEoOWUo69VI0FrjTng/0hYamuSc641lDjijTsRWmiLb4hu3TpvPBEFs
ZFHQoX9Pkrb7gnuwVMXXhhMuJAKgV4W21VI6BwU7D000b3MCAwEAAaOCAhUwggIR
MA4GA1UdDwEB/wQEAwIFoDAdBgNVHSUEFjAUBggrBgEFBQcDAQYIKwYBBQUHAwIw
DAYDVR0TAQH/BAIwADAdBgNVHQ4EFgQUuRNBJtTf+OmZZOaBSyVGz6C311UwHwYD
VR0jBBgwFoAUu7zDR6XkvKnGw6RyDBCNojXhyOgwVwYIKwYBBQUHAQEESzBJMCIG
CCsGAQUFBzABhhZodHRwOi8vcjEwLm8ubGVuY3Iub3JnMCMGCCsGAQUFBzAChhdo
dHRwOi8vcjEwLmkubGVuY3Iub3JnLzAbBgNVHREEFDASghBtYWlsLmJldXRoZW4u
bmV0MBMGA1UdIAQMMAowCAYGZ4EMAQIBMIIBBQYKKwYBBAHWeQIEAgSB9gSB8wDx
AHcAzPsPaoVxCWX+lZtTzumyfCLphVwNl422qX5UwP5MDbAAAAGVXj7JvgAABAMA
SDBGAiEAvXvb4JbfSwgwoTHmTb6Ejtwob3/usKqOaDHRLHLfeR4CIQDRuMBb36ds
r6Oq8VQ+4n3xVI+FhBvNVgh3G6X4hoIu8QB2AObSMWNAd4zBEEEG13G5zsHSQPaW
hIb7uocyHf0eN45QAAABlV4+ycEAAAQDAEcwRQIhAP+BUfcoR1MYbiBdj4aXpgnD
HhHjE2ZzlPPcOQSoh487AiBhCajocCKDoLfxsr4XF2rRL433lw5DpT9wnCdHNHJl
pDANBgkqhkiG9w0BAQsFAAOCAQEAyqN17RwpMKilEIhbygXuBbY8HT6s/mufH4ux
5ysx9NFw7ZRMj3Vg1jeDoo73CX3mDM0f2PynI+ERLcnoYdECZ/SkhUi6avC2oC4H
ppLpMW5RM6uRb0mUA/NOZxWyNz3UbXR3w9z6c7GUHXd9MQduqVQnnq1QyTjedyor
SqU25u7Xvhg+l+rR7vagW6L6pPflBmJmgQJsq3z7wALuLfrjSwxKGTe5zy4XwDb2
gQ7dElpzp7xaeZjrp5LzIhXfCAWFkGOin73dzALnKnjk+4vni3BFXPEf59zyWbau
cbjrInGv6QhtIuDIJFUMAWnz7q772FQsxAvQkE+A6lUzL+XTXA==
-----END CERTIFICATE-----
subject=CN=mail.beuthen.net
issuer=C=US, O=Let's Encrypt, CN=R10
---
No client certificate CA names sent
Peer signing digest: SHA256
Peer signature type: RSA-PSS
Server Temp Key: X25519, 253 bits
---
SSL handshake has read 3615 bytes and written 391 bytes
Verification: OK
---
New, TLSv1.3, Cipher is TLS_CHACHA20_POLY1305_SHA256
Protocol: TLSv1.3
Server public key is 4096 bit
This TLS version forbids renegotiation.
Compression: NONE
Expansion: NONE
No ALPN negotiated
Early data was not sent
Verify return code: 0 (ok)
---
---
Post-Handshake New Session Ticket arrived:
SSL-Session:
    Protocol  : TLSv1.3
    Cipher    : TLS_CHACHA20_POLY1305_SHA256
    Session-ID: D2774B1B82C27B90D084F848661DC7BDF33C604B6D932D76C7DCD4C796CE46A7
    Session-ID-ctx:
    Resumption PSK: 6FF5B072797551AB2501231C38F34F1B79AED4E7D4AC9A795CB3282ACDE8589D
    PSK identity: None
    PSK identity hint: None
    SRP username: None
    TLS session ticket lifetime hint: 604800 (seconds)
    TLS session ticket:
    0000 - 43 01 a6 73 98 0d c1 9b-f4 98 9d f2 b8 ff 1e 6f   C..s...........o
    0010 - 3c e0 44 7e 7f 13 11 3b-c7 47 eb 4b e6 17 2f 6a   <.D~...;.G.K../j
    0020 - bd ba 8c 71 2c eb 1f 86-d4 2e 15 cf 86 55 63 de   ...q,........Uc.
    0030 - 76 b7 cd ac a6 08 89 59-85 d6 0b 83 af d0 ce b9   v......Y........
    0040 - 24 84 32 fa 9c d7 1d da-c7 67 30 21 ba bc ff 67   $.2......g0!...g
    0050 - 50 d8 1e 0a c0 d6 6c 91-93 94 0f de 60 8b 74 f3   P.....l.....`.t.
    0060 - 06 5a 9f 29 45 46 87 52-a1                        .Z.)EF.R.

    Start Time: 1744908844
    Timeout   : 7200 (sec)
    Verify return code: 0 (ok)
    Extended master secret: no
    Max Early Data: 0
---
read R BLOCK

Is my assumption correct that this is a Caddy misconfiguration?


Caddyfile:
   layer4 {
      import /usr/local/etc/caddy/caddy.d/*.layer4global

      tcp/:993 {
         @451bca01-5c14-4866-8862-32d7285d0419 tls sni mail.beuthen.net

         route @451bca01-5c14-4866-8862-32d7285d0419 {
            tls
            proxy tcp/dms.beuthen.net:993 {
            }
         }
      }
   }


JSON Configuration:
    "layer4": {
      "servers": {
        "srv0": {
          "listen": [
            "tcp/:993"
          ],
          "routes": [
            {
              "handle": [
                {
                  "handler": "tls"
                },
                {
                  "handler": "proxy",
                  "upstreams": [
                    {
                      "dial": [
                        "tcp/dms.beuthen.net:993"
                      ]
                    }
                  ]
                }
              ],
              "match": [
                {
                  "tls": {
                    "sni": [
                      "mail.beuthen.net"
                    ]
                  }
                }
              ]
            }
          ]
        }
      }
    },


Thank you

I usually leave the only HA proxy threads alone :-)

Please don't on account of a silly comment. I'm sure your inputs would help.
p.s. I wish we could move support questions out of the Tutorials and FAQs section.

I am really sorry for the wrong forum - I didn't find a way to move it to the General questions. Can you please give me a hint how to do that maybe via PM? thanks

Hi All

I am struggling with the setup of HAProxy for IMAP/S (for HTTP, I use Caddy). In the HAProxy logs, all I can see is:

Informational   haproxy   Connect from <client IP>:61628 to <OPNsense WAN IP>:993 (public_imaps/TCP)

But my (MacOS Mail) client returns an error. I was told, that the issue is what I can see in the HAProxy config export:

backend pool_dms_imaps
    # health checking is DISABLED
    mode tcp
    balance source
    # stickiness
    stick-table type ip size 50k expire 30m 
    stick on src
    server dms_imaps <internal mail server name>:993 maxconn 100 ssl alpn h2,http/1.1 verify none send-proxy-v2 check-send-proxy

"alpn h2,http/1.1":
HAProxy, as I was told, somehow defaults to HTTP and that is why mail client access fails.

I use:
- Multiplexer Protocol = none in Real Server
- Mode = TCP in Backend Pool
- Type = TCP in Public Service

curl -vk https://<public DNS name>:993
* Host <public DNS name>:993 was resolved.
* IPv6: (none)
* IPv4: <OPNsense WAN IP>
*   Trying <OPNsense WAN IP>:993...
* Connected to <public DNS name> (<OPNsense WAN IP>) port 993
* ALPN: curl offers h2,http/1.1
* (304) (OUT), TLS handshake, Client hello (1):
* (304) (IN), TLS handshake, Server hello (2):
* (304) (IN), TLS handshake, Unknown (8):
* (304) (IN), TLS handshake, Certificate (11):
* (304) (IN), TLS handshake, CERT verify (15):
* (304) (IN), TLS handshake, Finished (20):
* (304) (OUT), TLS handshake, Finished (20):
* SSL connection using TLSv1.3 / AEAD-CHACHA20-POLY1305-SHA256 / [blank] / UNDEF
* ALPN: server accepted h2
* Server certificate:
*  subject: CN=<public DNS name>
*  start date: Mar  3 22:03:37 2025 GMT
*  expire date: Jun  1 22:03:36 2025 GMT
*  issuer: C=US; O=Let's Encrypt; CN=R10
*  SSL certificate verify ok.
* using HTTP/2
* [HTTP/2] [1] OPENED stream for https://<public DNS name>:993/
* [HTTP/2] [1] [:method: GET]
* [HTTP/2] [1] [:scheme: https]
* [HTTP/2] [1] [:authority: <public DNS name>:993]
* [HTTP/2] [1] [:path: /]
* [HTTP/2] [1] [user-agent: curl/8.7.1]
* [HTTP/2] [1] [accept: */*]

To those using HAProxy for IMAP/S. Can you please share what you have configured differently or am I misguided by assuming that "alpn h2,http/1.1" is causing my issue? Can this setup be configured in the UI or only via some config includes via SSH? I couldn't find any hint on that.

Thank you, Patric

Sorry, habe keine Benachrichtigung erhalten hierzu.

Nach meinem Verständnis müsste der Proxmox Hostname in meinem DNS gepflegt sein und über HTTP/HTTPS erreichbar sein. Das ist aber ein rein internes System und somit finde ich das nicht sehr elegant - bin aber damit noch recht unerfahren. Der Charme des ACME Clients in der OPNsense ist meiner Meinung nach, dass ich Zertifikate für im öffentlichen DNS unbekannte Hostnamen erstellen lassen kann und eben auch nur einen Ort für die ganze Landschaft habe.

Bin allerdings gerade daran, Caddy als reverse Proxy aufzusetzen und der wiederum möchte sich ja auch lieber selbst um die Zertifikate kümmern. Wobei mir dort aber auch nicht klar wäre, wie das in Caddy generierte Zertifikat weitergegeben wird an Proxmox und wie all das funktionieren soll mit einem Auth Server wie zB Authelia

[PS: Noch eine Frage zur Proxmox Automation. Die funktioniert bei mir auch nicht trotz exit code 0. Im letzten Jahr gab es glaube ich einen Fehler, der aber doch sicherlich längst behoben ist. Funktioniert das aktuell bei jemanden von Euch?]

Hallo zusammen

Eigentlich hatte ich gehofft, dass ich mit meiner ersten Frage unter General Discussions erfolgreicher bin. Nun ja, vielleicht ist sie entweder zu simple oder zu seltsam. Ich versuche es mal hier auf Deutsch und wäre den Sehenden unter Euch unglaublich dankbar für ein paar Tipps, wie ich dem Problem auf den Grund gehen kann.

Der ACME Client in der OPNsense (24.7.10_2) beschafft zuverlässig LE Zertifikate. Diese sollen intern weiterverteilt werden. Ich habe 2 Synology NAS Systeme (DSM DSM 7.2.2-72806) in unterschiedlichen VLANs. Bei einem funktioniert die Synology Automation ganz wunderbar, beim anderen nicht (s.u.). Ich habe auch erfolgreiche SFTP Automation mit dem jeweils selben Synology user getestet und komme auch von der OPNsense per SSH auf die Systeme. Meine Domäne ist öffentlich, die beiden NAS hostname sind aber nur im internen DNS (separater Container mit BIND9) bekannt.


2024-12-10T14:24:58   acme.sh   [Tue Dec 10 14:24:58 CET 2024] Error encountered while deploying.
2024-12-10T14:24:58   acme.sh   [Tue Dec 10 14:24:58 CET 2024] Error deploying for domain: <host>.<domain>
2024-12-10T14:24:58   acme.sh   [Tue Dec 10 14:24:58 CET 2024] Unable to authenticate to https://<host>.<domain>:5001, you may report this by providing full log with '--debug 3'.
2024-12-10T14:24:58   acme.sh   [Tue Dec 10 14:24:58 CET 2024] SynoToken
2024-12-10T14:24:58   acme.sh   [Tue Dec 10 14:24:58 CET 2024] Session ID
2024-12-10T14:24:58   acme.sh   [Tue Dec 10 14:24:58 CET 2024] error_code
2024-12-10T14:24:58   acme.sh   [Tue Dec 10 14:24:58 CET 2024] response
2024-12-10T14:24:58   acme.sh   [Tue Dec 10 14:24:58 CET 2024] ret='60'
...
2024-12-10T14:24:58   acme.sh   [Tue Dec 10 14:24:58 CET 2024] == Info: Host <host>.<domain>:5001 was resolved.
2024-12-10T14:24:58   acme.sh   [Tue Dec 10 14:24:58 CET 2024] Here is the curl dump log:
2024-12-10T14:24:58   acme.sh   [Tue Dec 10 14:24:58 CET 2024] Please refer to https://curl.haxx.se/libcurl/c/libcurl-errors.html for error code: 60
2024-12-10T14:24:58   acme.sh   [Tue Dec 10 14:24:58 CET 2024] _CURL='curl --silent --dump-header /var/etc/acme-client/home/http.header -L --trace-ascii /tmp/tmp.uMP3NA0rC9 -g '
2024-12-10T14:24:58   acme.sh   [Tue Dec 10 14:24:58 CET 2024] wget exists=127
2024-12-10T14:24:58   acme.sh   [Tue Dec 10 14:24:58 CET 2024] mktemp exists=0
2024-12-10T14:24:58   acme.sh   [Tue Dec 10 14:24:58 CET 2024] curl exists=0
2024-12-10T14:24:58   acme.sh   [Tue Dec 10 14:24:58 CET 2024] timeout=
2024-12-10T14:24:58   acme.sh   [Tue Dec 10 14:24:58 CET 2024] url='https://<host>.<domain>:5001/webapi/?api=SYNO.API.Auth&version=&method=login&format=sid&account=nushi&passwd=%40aF32aggHXrtKU&enable_syno_token=yes'
2024-12-10T14:24:58   acme.sh   [Tue Dec 10 14:24:58 CET 2024] GET
2024-12-10T14:24:57   acme.sh   [Tue Dec 10 14:24:57 CET 2024] _hex_str=' 40 61 46 33 32 61 67 67 48 58 72 74 4b 55'
2024-12-10T14:24:57   acme.sh   [Tue Dec 10 14:24:57 CET 2024] _url_encode
2024-12-10T14:24:57   acme.sh   [Tue Dec 10 14:24:57 CET 2024] od exists=0
2024-12-10T14:24:57   acme.sh   [Tue Dec 10 14:24:57 CET 2024] _hex_str=' 6e 75 73 68 69'
2024-12-10T14:24:57   acme.sh   [Tue Dec 10 14:24:57 CET 2024] _url_encode
2024-12-10T14:24:57   acme.sh   [Tue Dec 10 14:24:57 CET 2024] od exists=0
2024-12-10T14:24:57   acme.sh   [Tue Dec 10 14:24:57 CET 2024] Logging into <host>.<domain>:5001...
2024-12-10T14:24:57   acme.sh   [Tue Dec 10 14:24:57 CET 2024] api_version
2024-12-10T14:24:57   acme.sh   [Tue Dec 10 14:24:57 CET 2024] api_path
2024-12-10T14:24:57   acme.sh   [Tue Dec 10 14:24:57 CET 2024] response
2024-12-10T14:24:57   acme.sh   [Tue Dec 10 14:24:57 CET 2024] ret='60'

Zwischenzeitlich habe ich das selbstsignierte Zertifikat des NAS ersetzt durch das LE Zertifikat, der Fehler ist aber noch derselbe. Viel zu rc 60 habe ich auch nicht gefunden - erst recht nicht, was genau das Problem ist

All I can find about return code 60 is this:
CURLE_PEER_FAILED_VERIFICATION (60): The remote server's SSL certificate or SSH fingerprint was deemed not OK. This error code has been unified with CURLE_SSL_CACERT since 7.62.0. Its previous value was 51.

Vielen Dank für Eure Zeit!
Patric

PS: Noch eine Frage zur Proxmox Automation. Die funktioniert bei mir auch nicht trotz exit code 0. Im letzten Jahr gab es glaube ich einen Fehler, der aber doch sicherlich längst behoben ist. Funktioniert das aktuell bei jemanden von Euch?

Dear All

I am struggling with ACME client certification deployment to Synology. What is so strange to me is, that it works for one Synology NAS but not for the other.

My setup:
- OPNsense on 24.7.10_2
- 2 * Synology on DSM 7.2.2-72806
- Both NAS systems are in different VLANs
- SSH access is fine from OPNsense root to relevant NAS user

I have set Log Level to debug3 in ACME Client settings but I do not see more than the following.

My domain is public but the hostnames of these 2 NAS systems are not. I am running BIND9 as DNS for my local network

I successfully tested SFTP automation to both NAS systems with the same Synology users. OTP are not configured.



2024-12-08T09:31:06   acme.sh   [Sun Dec 8 09:31:06 CET 2024] Error encountered while deploying.
2024-12-08T09:31:06   acme.sh   [Sun Dec 8 09:31:06 CET 2024] Error deploying for domain: <host>.<domain>
2024-12-08T09:31:06   acme.sh   [Sun Dec 8 09:31:06 CET 2024] Unable to authenticate to https://<host>.<domain>:5001, you may report this by providing full log with '--debug 3'.
2024-12-08T09:31:06   acme.sh   [Sun Dec 8 09:31:06 CET 2024] SynoToken
2024-12-08T09:31:06   acme.sh   [Sun Dec 8 09:31:06 CET 2024] Session ID
2024-12-08T09:31:06   acme.sh   [Sun Dec 8 09:31:06 CET 2024] error_code
2024-12-08T09:31:06   acme.sh   [Sun Dec 8 09:31:06 CET 2024] response
2024-12-08T09:31:06   acme.sh   [Sun Dec 8 09:31:06 CET 2024] ret='60'

Meanwhile I manually uploaded the LE certificate and set it as standard cert (I also deleted the self signed Synology cert)

All I can find about return code 60 is this:
CURLE_PEER_FAILED_VERIFICATION (60): The remote server's SSL certificate or SSH fingerprint was deemed not OK. This error code has been unified with CURLE_SSL_CACERT since 7.62.0. Its previous value was 51.

Any advice that helps me to identify the misconfiguration is highly appreciated.