Hey all,
This is on an OPNsense v25.7.6 firewall.
I've created probably over a hundred wildcard certs in my career, so this process is not unfamiliar to me.
In this case, I'm renewing a wildcard certificate from Sectigo for the next year.
This certificate will be used by 15-20 different servers internally that have public facing services, like remote.example.com, wiki.example.com, and on and on.
After I get Sectigo's new CA and intermediate certificates imported in System > Trust > Authorities (now with a new 2036-03-21 expiration date), I import my wildcard certificate in System > Trust > Certificates.
I do the whole self-signed vs signed by a real CA dance that can be referenced here (https://forum.opnsense.org/index.php?topic=41943.0, reply #9), and still seems to be a bug in OPNsense.
I now have a certificate for *.example.com listing the Sectigo CA above as the Issuer and an expiration date of 2026-11-05. So far, so good.
I go to Services > HAProxy > Settings > Virtual Services > Public Services > ps_https_frontend and click Edit.
I leave "Enable SSL Offloading" selected and add my new certificate to the Certificates section, then Save and Test Syntax. I get:
HAProxy configtest found critical errors
[NOTICE] (66728) : haproxy version is 3.0.12-21a355d
[NOTICE] (66728) : path to executable is /usr/local/sbin/haproxy
[ALERT] (66728) : config : parsing [/usr/local/etc/haproxy.conf.staging:62] : 'bind 127.4.4.3:443' in section 'frontend' :
'crt-list' : error processing line 1 in file '/tmp/haproxy/ssl/6550339315a041.13247647.certlist' : inconsistencies between private key and certificate loaded '/tmp/haproxy/ssl/6907e83ad16a7.pem'.
unable to load SSL private key into SSL Context '/tmp/haproxy/ssl/6907e83ad16a7.pem': passed a null parameter.
[ALERT] (66728) : config : Error(s) found in configuration file : /usr/local/etc/haproxy.conf.staging
[ALERT] (66728) : config : Fatal errors found in configuration.
OK, so it doesn't like something. I drop to an SSH terminal on my firewall and run the following command:
cat /tmp/haproxy/ssl/6907e83ad16a7.pem
My PEM looks perfect. It starts with my Certificate (*.example.com), followed by my Private Key, followed by Sectigo's CA Certificate and Intermediate Certificate, just as I pasted them into the CA section above. All of the begin and end tags are on different lines throughout the file. No spacing issues that I can see.
If I run this:
openssl x509 -in /tmp/haproxy/ssl/6907e83ad16a7.pem -noout -text
My certificate data looks great. If I take the public certificate and private key and run them against this:
openssl x509 -noout -modulus -in 2025-2026.star_example_com.crt | openssl md5
openssl rsa -noout -modulus -in 2025-2026.star_example_com.key | openssl md5
The results are:
MD5(stdin)= 54....187
MD5(stdin)= 54....187
They're a match.
So, I'm out of troubleshooting steps. My certificate looks great as far as I can tell, but I cannot seem to use it with HAProxy, and that's pretty much a deal breaker. My current certificate expires on the 4th. Anyone have any ideas?
Thanks in advance!
I'm going to mark this SOLVED here in a minute. Here's what worked:
I went into System > Trust > Authorities and edited the one I put in there last night for Sectigo. Cleared out the Certificate Data field and made sure there was nothing in there... no newline/carriage returns, nothing there. Then I opened the ca-bundle file from Sectigo, and Control-A, Control-C, Control-V'ed it into the Certificate data field. Click Save.
Next I went to System > Trust > Certificates and Edited the new certificate. I cleared out all of the certificate fields as above.... no anything in there that doesn't belong. Then I opened each file: cert.pem/cert.crt, cert.key, cert.csr. Again with Control-A, Control-C, Control-V for each of them into the appropriate fields.
Here's the "magic": After this, I went to the bottom of each field and made certain there was a single carriage return/newline after the "-----". Just one. No more, no less. Save the certificate.
Add it to HAProxy and test. It suddenly works.
Why does this matter on HAProxy and not anywhere else? I have no idea. Why on OPNsense and possibly nowhere else? Again, no idea. All I can tell you is, the fullchain.pem passes all tests with flying colors, but it does not work without the above solution here.
As usual, I provide this information with my standard warranty on forums. If this worked for you, great. If it didn't work for you, well, you got what you paid for. Hopefully it's the former. But it's worth providing the information to everyone if I'm going to post the question and I find an answer that works for me.