Messages

Hi

Today, in one of our production instances, there was some issue with IPsec. Dashboard was accessed to check IPsec status and logs, plus firewall logs. After about 4m, dashboard stopped working with "503 Service Unavailable".

Instance was shortly rebooted, and is since then working as expected. As part of RCA, i found this in lighttpd.logs

<27>1 2022-12-16T08:42:41+02:00 vpn-server.localdomain lighttpd 97433 - [meta sequenceId="1"] (gw_backend.c.283) establishing connection failed: socket: unix:/tmp/php-fastcgi.socket-1: Connection refused
<27>1 2022-12-16T08:42:44+02:00 vpn-server.localdomain lighttpd 97433 - [meta sequenceId="2"] (gw_backend.c.360) gw-server re-enabled: unix:/tmp/php-fastcgi.socket-1  0 /tmp/php-fastcgi.socket
...
<27>1 2022-12-16T08:45:44+02:00 vpn-server.localdomain lighttpd 97433 - [meta sequenceId="122"] (gw_backend.c.360) gw-server re-enabled: unix:/tmp/php-fastcgi.socket-1  0 /tmp/php-fastcgi.socket
<27>1 2022-12-16T08:45:45+02:00 vpn-server.localdomain lighttpd 97433 - [meta sequenceId="123"] (gw_backend.c.283) establishing connection failed: socket: unix:/tmp/php-fastcgi.socket-1: Connection refused
<27>1 2022-12-16T08:45:45+02:00 vpn-server.localdomain lighttpd 97433 - [meta sequenceId="124"] (gw_backend.c.283) establishing connection failed: socket: unix:/tmp/php-fastcgi.socket-0: Connection refused
<27>1 2022-12-16T08:45:45+02:00 vpn-server.localdomain lighttpd 97433 - [meta sequenceId="125"] (gw_backend.c.993) all handlers for /api/api.php?digest=fd2e7a3bc8d4bbf00c79b9967f55f464&limit=1000 on .php are down.
<27>1 2022-12-16T08:45:48+02:00 vpn-server.localdomain lighttpd 97433 - [meta sequenceId="126"] (gw_backend.c.360) gw-server re-enabled: unix:/tmp/php-fastcgi.socket-1  0 /tmp/php-fastcgi.socket
<27>1 2022-12-16T08:45:48+02:00 vpn-server.localdomain lighttpd 97433 - [meta sequenceId="127"] (gw_backend.c.360) gw-server re-enabled: unix:/tmp/php-fastcgi.socket-0  0 /tmp/php-fastcgi.socket

It is a virtual instance in an openstack platform, with 1CPU, 2GB of RAM, 150GB UFS. Currently on 22.7.7_1, no additional packages installed. Only serving as an IPsec gateway, with 12 tunnels installed.

Any ideas?

Thanks

In case someone hits this, I found an alternative solution.

Using IKEv2 + EAP-RADIUS on OPNsense side, and then EAP-MD5 on client side, does seem to work. Disadvantage is that a CA needs to be pushed to the client, instead of using PSK only.

Client ipsec.conf looks like this now

Code Select Expand

conn opnsense
      auto=start
      keyexchange=ikev2
      ike=aes256-sha256-modp2048
      esp=aes256-sha256-modp2048
      leftid=rwclient
      leftauth=eap-md5
      leftsourceip=%modeconfig
      leftsendcert=no
      right=<opnsense pub ip>
      rightid=rwserver
      rightsubnet=10.99.0.0/24
      closeaction=restart
      dpdaction=restart
      keyingtries=%forever

ipsec.secrets

Code Select Expand

rwclient : EAP "passw0rd"

And OPNsense IPsec log

2022-11-11T13:03:30   Informational   charon   07[IKE] <con2|398> assigning virtual IP 192.168.10.99 to peer 'rwclient'   
2022-11-11T13:03:30   Informational   charon   07[IKE] <con2|398> peer requested virtual IP %any

Still not sure if my original attempt is unsupported, not implemented or broken. I got this hint when i realized that the Framed-IP-Address is documented under strongswan's eap-radius plugin: https://docs.strongswan.org/docs/5.9/plugins/eap-radius.html#_radius_attribute_forwarding

[radius-server]

Hi all

Im trying to get and IPsec roadwarrior setup work with static IPs. After some investigation, i found hints that this should be possible using RADIUS's Framed-IP-Address attribute. However, after a bunch of trial and error, i havent been able to get it working.

The setup:
radius-server
freeradius 3.0.12 on Debian 10

Code Select Expand

rwclient Cleartext-Password := "passw0rd", Simultaneous-Use := "1"
Framed-IP-Address = 192.168.10.99,
Framed-IP-Netmask = 255.255.255.255,
Framed-Route = "10.99.0.0/24 192.168.10.1 1"


OPNsense 22.7.6-amd64
WAN 10.254.1.5/24, behind NAT
LAN 10.99.0.254/24

Mobile clients

Code Select Expand


Enabled
Backend: radius-server
everything else UNSET


Phase1

Code Select Expand


Respond only
IKEv1 main
Mutual PSK + Xauth
IP address identifier
<psk>
AES256 SHA256 PFS14
Lifetime 28800
everything else UNSET


Phase2

Code Select Expand


IPv4 tunnel
LAN subnet
AES256 SHA256 PFS14
Lifetime 3600
everything else UNSET



Client
strongswan 5.9.1 on Debian 11
eth0 IP 192.168.1.105/24

ipsec.conf

Code Select Expand


conn opnsense
  keyexchange=ikev1
  aggressive=no
  ike=aes256-sha256-modp2048
  esp=aes256-sha256-modp2048
  auto=start
  authby=xauthpsk
  leftid=rwclient
  leftsourceip=%modeconfig
  right=<opnsense pub ip>
  rightid=<opnsense pub ip>
  rightsubnet=10.99.0.0/24


ipsec.secrets

Code Select Expand


<opnsense pub ip> : PSK "<psk>"
rwclient: XAUTH "passw0rd"



From OPNsense ipsec log, you can see that peer requests IP, but server does not return one:

2022-11-01T14:35:55   Informational   charon   05[ENC] <con2|8> generating INFORMATIONAL_V1 request 1520615772 [ HASH N(INVAL_ID) ]   
2022-11-01T14:35:55   Informational   charon   05[IKE] <con2|8> no matching CHILD_SA config found for 192.168.1.105/32 === 10.99.0.0/24   
2022-11-01T14:35:55   Informational   charon   05[ENC] <con2|8> parsed QUICK_MODE request 1074724946 [ HASH SA No KE ID ID ]   
2022-11-01T14:35:55   Informational   charon   05[NET] <con2|8> received packet: from 217.140.xxx.xxx[46716] to 10.254.1.5[4500] (460 bytes)   
2022-11-01T14:35:55   Informational   charon   05[NET] <con2|8> sending packet: from 10.254.1.5[4500] to 217.140.xxx.xxx[46716] (76 bytes)   
2022-11-01T14:35:55   Informational   charon   05[ENC] <con2|8> generating TRANSACTION response 1061020512 [ HASH CP ]   
2022-11-01T14:35:55   Informational   charon   05[IKE] <con2|8> no virtual IP found for %any requested by 'rwclient'   
2022-11-01T14:35:55   Informational   charon   05[IKE] <con2|8> peer requested virtual IP %any

If i add an IPv4 pool to the mobile clients settings page, like 192.168.99.0/24, then phase2 is established, but with IP from the pool:

2022-11-01T14:40:10   Informational   charon   11[IKE] <con2|9> CHILD_SA con2{22} established with SPIs cc662891_i cc66c029_o and TS 10.99.0.0/24 === 192.168.99.1/32   
2022-11-01T14:40:10   Informational   charon   11[ENC] <con2|9> parsed QUICK_MODE request 323113376 [ HASH ]   
2022-11-01T14:40:10   Informational   charon   11[NET] <con2|9> received packet: from 217.140.xxx.xxx[46716] to 10.254.1.5[4500] (76 bytes)   
2022-11-01T14:40:10   Informational   charon   11[NET] <con2|9> sending packet: from 10.254.1.5[4500] to 217.140.xxx.xxx[46716] (460 bytes)   
2022-11-01T14:40:10   Informational   charon   11[ENC] <con2|9> generating QUICK_MODE response 323113376 [ HASH SA No KE ID ID ]   
2022-11-01T14:40:10   Informational   charon   11[CFG] <con2|9> selected proposal: ESP:AES_CBC_256/HMAC_SHA2_256_128/MODP_2048/NO_EXT_SEQ   
2022-11-01T14:40:10   Informational   charon   11[ENC] <con2|9> parsed QUICK_MODE request 323113376 [ HASH SA No KE ID ID ]   
2022-11-01T14:40:10   Informational   charon   11[NET] <con2|9> received packet: from 217.140.xxx.xxx[46716] to 10.254.1.5[4500] (460 bytes)   
2022-11-01T14:40:10   Informational   charon   11[NET] <con2|9> sending packet: from 10.254.1.5[4500] to 217.140.xxx.xxx[46716] (188 bytes)   
2022-11-01T14:40:10   Informational   charon   11[ENC] <con2|9> generating TRANSACTION response 3117777291 [ HASH CPRP(ADDR SUBNET SUBNET SUBNET SUBNET U_SPLITINC U_SPLITINC U_SPLITINC U_SPLITINC) ]   
2022-11-01T14:40:10   Informational   charon   11[IKE] <con2|9> assigning virtual IP 192.168.99.1 to peer 'rwclient'   
2022-11-01T14:40:10   Informational   charon   11[CFG] <con2|9> assigning new lease to 'rwclient'   
2022-11-01T14:40:10   Informational   charon   11[IKE] <con2|9> peer requested virtual IP %any

I have confirmed that the RADIUS server is returning the correct data, using OPNsense's System > Access > Tester

Code Select Expand

User: rwclient authenticated successfully.
This user is a member of these groups:


Attributes received from server:
Framed-IP-Address => 192.168.10.99
Framed-IP-Netmask => 255.255.255.255
Framed-Route => 10.99.0.0/24 192.168.10.1 1


So, what i want to know is if i am doing something wrong, or if this is a bug/non-implemented in OPNsense

Thanks