Unable to stablish first IPSec VPN

[cookiemonster]

I have never done the esp-tls setup.

From what I understand from a high level view, you need 3 certificates:

Self signed root
root signed leaf (server use)
root signed leaf (client use)

Best no lets encrypt anywhere.

The server and the client both need the signed root.

The server needs the server certificate. The client /must/ use an FQDN to contact the von server, and the FQDN must be in the server certificate, as SAN (subject alternative name) otherwise no match.

The client needs the client certificate. The eap username configured in the opnsense must match with the common name in the client certificate, otherwise no match.

In the remote and local authentication these certificates have to be set.

Im not 100% sure here but thats what I imagine.

Exactly what I have Cedrik, and have done various times to ensure I weed out any mistakes but I keep failing.
The only element I'd like to validate is this one, where I feel I have a misconfiguration OR OPN is not matching a cert where it should:

The client needs the client certificate. The eap username configured in the opnsense must match with the common name in the client certificate, otherwise no match.

Where is that done? The remote section in the docs have no users to use
You cannot view this attachment.
I have been using the client cert (.p12 bundle of cert and private key & password protected), with the client also of course having the public cert of the CA installed and made trusted. The CN on this client (leaf) cert and SAN for DNS also is signed by the CA on OPN. But you can still see that the IPSec failure log still says "certificate unknown"

Code Select Expand

2025-08-27T23:24:39 Informational charon 14[MGR] <1c9aaddd-79af-41d2-9b2b-db85ae839384|33> checkin and destroy IKE_SA 1c9aaddd-79af-41d2-9b2b-db85ae839384[33]
2025-08-27T23:24:39 Informational charon 03[NET] sending packet: from 92.26.121.196[4500] to 192.168.5.235[4500]
2025-08-27T23:24:39 Informational charon 14[NET] <1c9aaddd-79af-41d2-9b2b-db85ae839384|33> sending packet: from 92.26.121.196[4500] to 192.168.5.235[4500] (80 bytes)
2025-08-27T23:24:39 Informational charon 14[ENC] <1c9aaddd-79af-41d2-9b2b-db85ae839384|33> generating IKE_AUTH response 5 [ EAP/FAIL ]
2025-08-27T23:24:39 Informational charon 14[IKE] <1c9aaddd-79af-41d2-9b2b-db85ae839384|33> EAP method EAP_TLS failed for peer penguin-vpn1
2025-08-27T23:24:39 Informational charon 14[TLS] <1c9aaddd-79af-41d2-9b2b-db85ae839384|33> received fatal TLS alert 'certificate unknown'
2025-08-27T23:24:39 Informational charon 14[ENC] <1c9aaddd-79af-41d2-9b2b-db85ae839384|33> parsed IKE_AUTH request 5 [ EAP/RES/TLS ]
2025-08-27T23:24:39 Informational charon 14[NET] <1c9aaddd-79af-41d2-9b2b-db85ae839384|33> received packet: from 192.168.5.235[4500] to 92.26.121.196[4500] (96 bytes)
2025-08-27T23:24:39 Informational charon 14[MGR] IKE_SA 1c9aaddd-79af-41d2-9b2b-db85ae839384[33] successfully checked out
2025-08-27T23:24:39 Informational charon 14[MGR] checkout IKEv2 SA by message with SPIs 65ce1e4934bf1455_i c22f661e67a203da_r
2025-08-27T23:24:39 Informational charon 02[NET] waiting for data on sockets
2025-08-27T23:24:39 Informational charon 02[NET] received packet: from 192.168.5.235[4500] to 92.26.121.196[4500]
2025-08-27T23:24:39 Informational charon 03[NET] sending packet: from 92.26.121.196[4500] to 192.168.5.235[4500]
2025-08-27T23:24:39 Informational charon 14[MGR] <1c9aaddd-79af-41d2-9b2b-db85ae839384|33> checkin of IKE_SA successful
2025-08-27T23:24:39 Informational charon 14[MGR] <1c9aaddd-79af-41d2-9b2b-db85ae839384|33> checkin IKEv2 SA 1c9aaddd-79af-41d2-9b2b-db85ae839384[33] with SPIs 65ce1e4934bf1455_i c22f661e67a203da_r
2025-08-27T23:24:39 Informational charon 14[NET] <1c9aaddd-79af-41d2-9b2b-db85ae839384|33> sending packet: from 92.26.121.196[4500] to 192.168.5.235[4500] (832 bytes)
2025-08-27T23:24:39 Informational charon 14[ENC] <1c9aaddd-79af-41d2-9b2b-db85ae839384|33> generating IKE_AUTH response 4 [ EAP/REQ/TLS ]
2025-08-27T23:24:39 Informational charon 14[ENC] <1c9aaddd-79af-41d2-9b2b-db85ae839384|33> parsed IKE_AUTH request 4 [ EAP/RES/TLS ]
2025-08-27T23:24:39 Informational charon 14[NET] <1c9aaddd-79af-41d2-9b2b-db85ae839384|33> received packet: from 192.168.5.235[4500] to 92.26.121.196[4500] (80 bytes)
2025-08-27T23:24:39 Informational charon 14[MGR] IKE_SA 1c9aaddd-79af-41d2-9b2b-db85ae839384[33] successfully checked out
2025-08-27T23:24:39 Informational charon 14[MGR] checkout IKEv2 SA by message with SPIs 65ce1e4934bf1455_i c22f661e67a203da_r
2025-08-27T23:24:39 Informational charon 02[NET] waiting for data on sockets
2025-08-27T23:24:39 Informational charon 02[NET] received packet: from 192.168.5.235[4500] to 92.26.121.196[4500]
2025-08-27T23:24:39 Informational charon 03[NET] sending packet: from 92.26.121.196[4500] to 192.168.5.235[4500]
2025-08-27T23:24:39 Informational charon 14[MGR] <1c9aaddd-79af-41d2-9b2b-db85ae839384|33> checkin of IKE_SA successful
2025-08-27T23:24:39 Informational charon 14[MGR] <1c9aaddd-79af-41d2-9b2b-db85ae839384|33> checkin IKEv2 SA 1c9aaddd-79af-41d2-9b2b-db85ae839384[33] with SPIs 65ce1e4934bf1455_i c22f661e67a203da_r
2025-08-27T23:24:39 Informational charon 14[NET] <1c9aaddd-79af-41d2-9b2b-db85ae839384|33> sending packet: from 92.26.121.196[4500] to 192.168.5.235[4500] (1104 bytes)
2025-08-27T23:24:39 Informational charon 14[ENC] <1c9aaddd-79af-41d2-9b2b-db85ae839384|33> generating IKE_AUTH response 3 [ EAP/REQ/TLS ]And that is the CN on the cert
You cannot view this attachment.

So, is there another place I must put the user on ?

[cookiemonster]

The symlinks are hash values of the content of the installed certificates. Each symlink should point to a valid file. There is a hash script to update/produce these, but of course in an appliance context all of this should "just work" without any orphaned ones.

For reference: https://docs.openssl.org/3.5/man1/openssl-rehash/#description

That is also very good information thank you Patrick. I definitively have sims to non-existing ones. No idea why. I haven't had any error messages when creating the CA and the other certs.
I'll have a read to see if I should rehash them.

[Monviech (Cedrik)]

I dont know how eap-tls works completely.

The part that I ran productively in that documentation is the mschapv2 setup, which does not beed client certificates and is much simpler to setup and troubleshoot.

Otherwise please just use openvpn or wireguard.

Im using openvpn with viscosity on a mac and never looked back, ipsec is just pain compared to that. And it continues to work after macos updates, unlike ipsec where they change something convoluted about the ciphers they accept and bam it just doesnt work anymore.

I will most likely not spend any time on this, sorry.

[cookiemonster]

Ok that's fair if you don't use it. Thanks for the inputs you've made to this thread.

[bimbar]

If it's site2site, use a suitably long PSK.

IPSec is bad enough without using certificates, unfortunately.

Speaking as the one who does the level 3 ipsec support at my employer.

[cookiemonster]

thanks. It is a static "site" bimbar but technically is a road warrior setup. It is for my son to be able to connect to home now that he has moved out, so that he can backup to the home NAS.
For that I want to issue him certs he can then use on an apple mac but also on his iphone from time to time. There are other reasons I want to have this setup even if I have a wireguard setup working fine.
So I am mis-classifying it as S2S. It is road warrior.
Pesky certificates are not seemingly being accepted/found by OPN despite being setup as per documentation.

Any thoughts on what else to check, I'll be grateful.

[Monviech (Cedrik)]

Use OpenVPN, it really works flawlessly with any apple devices, and it also uses client certificates.

It is well documented and most businesses use it (and not ipsec) for these kinds of setups.

[cookiemonster]

Problem is Cedrik that I also need to be able to connect to home network from time to time when using work devices. OpenVPN is not allowed but IPSec is. Not my policies.
I appreciate the input, I do. However I fail to understand why core functionality seems to not work despite following the documentation, and suggestions are to not use it.

[Monviech (Cedrik)]

These IPsec roadwarrior setup examples are more in community support scope, they are incredibly hard to troubleshoot without doing remote support. And even then its crawling logs and turning knobs until it finally works.

A good example to start with would be the MSCHAPv2 guide, I used that on MacOS before with the NCP client, productively with a business client in my previous job for around 2 years.

I needed around 2 weeks to get a stable setup, thats when I wrote the documentation for it (while I was still a community member).

The EAP-TLS example is a creation by a different user in the forum, adapted by me. I followed that discussion and saw how different iOS or MacOS version changed behavior without prior notice, and it stopped to work for that user on quite some occasions.

If the EAP-TLS example does not work, please create an issue in the docs repository and point out which steps are wrong or unclear. But verifying and testing this takes lots of time, as you probably noticed yourself by now. It is just a very complex setup that not a lot of people use right now in the connections GUI.

[cookiemonster]

Understood. Thank you Cedrik.