A couple of firewalls experienced site-to-site IPsec IKEv2 issues after upgrading to 24.7.2. We reverted back to 24.7.1 and it seems to return to normal.
The IPsec connections terminate at a FortiGate, and the connections are either lost and recovered by rebooting the OpnSense, or the connection is lost completely (a reboot doesn't reestablish the connection).
Returning to 24.7.1 has corrected this.
Sali,
Can you share the difference of /usr/local/etc/strongswan.conf between version 24.7.1 and 24.7.2? (diff -u)
Privately is fine too: franco@opnsense.org
Thanks,
Franco
Hi Franco,
I haven't captured the 24.7.2 version of the file because terminal logging was off, but I compared it to one of the other firewalls still running 24.7.2. There may be slight differences in how the VPN is configured. The 02-strongswan.conf is still running 24.7.2.
~ @ ctmac01(xxxxxxx): diff -u *strongswan.conf
--- 01-strongswan.conf 2024-08-23 15:24:40
+++ 02-strongswan.conf 2024-08-23 15:23:12
@@ -9,9 +9,27 @@
init_limit_half_open = 1000
ignore_acquire_ts = yes
syslog {
- identifier = charon
+ ike_name = yes
+ log_level = no
daemon {
- ike_name = yes
+ app = 1
+ asn = 1
+ cfg = 1
+ chd = 1
+ dmn = 1
+ enc = 1
+ esp = 1
+ ike = 1
+ imc = 1
+ imv = 1
+ job = 1
+ knl = 1
+ lib = 1
+ mgr = 1
+ net = 1
+ pts = 1
+ tls = 1
+ tnc = 1
}
}
install_routes = no
@@ -19,4 +37,3 @@
}
}
-include strongswan.opnsense.d/*.conf
Regards,
Jaap
+1 for me as well after upgrading, IPSEC tunnels drop after a short period of time, service restart addresses the issue. Just a a FYI, if you need logs, glad to provide.
Thanks
Thanks so far. It's inconclusive unfortunately. Could someone provide the diff between version for the same system without modified settings?
Does anyone use strongswan.opnsense.d/*.conf overrides affected here?
Thanks,
Franco
I can downgrade one of the other firewalls tonight from 24.7.2 to 24.7.1 while saving the conf files before and after. We have about 15 remaining OPNsense that I think are now running 24.7.2. They all have IPsec connections with our Fortigate. The others are not (yet) misbehaving.
Please let me know if you need other files before and after to compare.
Correction, 8 devices have upgraded to 24.7.2.
To address this issue I enabled DPD:
We are having similar issues
Upgraded to 24.7.2 and IPSEC VPNs are having issues. When will this be resolved????
Just verified:
24.7.2 <--> 24.7.2 does NOT work
24.7.2 <--> 24.1.5 Still working
Will not be upgrading the 24.1.5 firewall anytime soon.
What is the issue and when will it be resolved? The 24.7.x line appears to be quite buggy.
Quote from: franco on August 23, 2024, 03:12:29 PM
Can you share the difference of /usr/local/etc/strongswan.conf between version 24.7.1 and 24.7.2? (diff -u)
Privately is fine too: franco@opnsense.org
Quote from: franco on August 23, 2024, 04:02:19 PM
Does anyone use strongswan.opnsense.d/*.conf overrides affected here?
Cheers,
Franco
Quote from: franco on August 23, 2024, 04:02:19 PM
Does anyone use strongswan.opnsense.d/*.conf overrides affected here?
Negative. Also, if IPsec is unavoidable, we always use DPD. It's working fine with 24.7.2
Quote from: dwoodroofe on August 25, 2024, 07:15:10 AM
What is the issue and when will it be resolved?
You know, that's what people here are trying to determine. Posts like yours are useless for that purpose.
Hello, I'd like to add to this by describing the symptoms experienced on my testbed platform. I have a number of virtual Opensense VMs interconnected via IPSEC which have worked fine up until 24.7.2.
However, on all of my 24.7.2 machines I now see in firewall logs ESP traffic dropping into the default WAN deny rule and so tunnels, while establishing, are passing no traffic.
It would appear that ESP previously covered by automatic rules generated upon enabling IPSEC has changed under 24.7.2. See attachment of log screenshot
What I have tested is a rollback to 24.7.1 by either restore of VM from backup or an interactive "opnsense-revert -r 24.7.1 opnsense". In both cases this fixes the issue and restores IPSEC traffic flow correctly.
Whilst not highly technical I hope description of symptoms helps somebody clevererer figure out any perceived issue with 24.7.2.
Thank you for Opnsense!
Cannot modify original message, wanted to add another test on a 24.7.2 prior to restore;
On the WAN interface I created a firewall rule to pass ESP traffic from source ip to wan interface, in my head mimicking what the automatically created rules might be doing.
This worked (definitely on a 24.7.2 VM) and permitted traffic to pass but I'd have to do that individually for all IPSEC connections on all WAN interfaces.
If somebody really wanted to stay on 24.7.2 and has only a handful of connections this could be a workaround until addressed by Opnsense gurus.
Regarding the firewall rules, reading the documentation is always helpful. https://docs.opnsense.org/manual/vpnet.html#firewall-rules
Those "magic" ones have not been there for quite a while except for the legacy tunnels, where it can be optionally enabled.
Quote from: monkfish on August 25, 2024, 11:47:28 AM
but I'd have to do that individually for all IPSEC connections on all WAN interfaces.
Or use the interface groups. And that's exactly what you have to do with any other firewall rules, no idea why should IPsec be treated specially.
Thanks for that was aware, however these machines have been around for a long time and were therefore probably built when the defaults referred to were set. They have certainly never been unset automatically by any particular update - until 24.7.2.
Sharing my particular symptom and fix.
I had here a similar issue with some firewalls which are updated to 24.7.2 (not all)
The issue is between a fw with 24.1.10 and a fresh updated 24.7.2 .
Because first i think thats again a issue on our provider (they hat a bgb router which don't terminate some sessions correctly). So i disabled on both firewalls the corresponding ipsec rules and re-enable them after half an hour. On the status page of both firewalls the tunnel is "installed" with the correct subnets, but no traffic goes through the tunnel.
on a host on the 24.1.10 net i can start a ping through the tunnel i see also in the log/tcpdump that there is traffic which goes through enc0 but tcpdump on the 24.7.2 on enc0 don't see any traffic. The intresting part is also that under firewall rules the ipsec part is completly missing.
As mentioned under https://forum.opnsense.org/index.php?topic=28326.0 i can see the rules when i call the rules directly. but disable ipsec and re-enabled doesn't help. currently i cannot reboot the 24.7.2.
Edit, Short Update:
If a edit a Rule then the IPSEC Block appears again. And Traffic goes through the VPN Tunnel when i manually add the rules which are previously created via auto rules.
Quote from: franco on August 25, 2024, 10:24:47 AM
Quote from: franco on August 23, 2024, 03:12:29 PM
Can you share the difference of /usr/local/etc/strongswan.conf between version 24.7.1 and 24.7.2? (diff -u)
Privately is fine too: franco@opnsense.org
Quote from: franco on August 23, 2024, 04:02:19 PM
Does anyone use strongswan.opnsense.d/*.conf overrides affected here?
Cheers,
Franco
I found one FW which i don't updated to 24.7.2 because they lost internet connection after reboot, and stuck this weekend on 24.7.1. Your mentioned config is complete different. Because no confidential data is inside /usr/local/etc/strongswan.conf i paste them here.
FW 24.7.1 with "Automatically generated rules (end of ruleset)" on WAN:
# Automatically generated, please do not modify
starter {
load_warning = no
}
charon {
threads = 16
ikesa_table_size = 32
ikesa_table_segments = 4
init_limit_half_open = 1000
ignore_acquire_ts = yes
syslog {
identifier = charon
daemon {
ike_name = yes
}
}
install_routes = no
plugins {
}
}
include strongswan.opnsense.d/*.conf
FW 24.7.2 without "Automatically generated rules (end of ruleset)" on WAN:
# Automatically generated, please do not modify
starter {
load_warning = no
}
charon {
threads = 16
ikesa_table_size = 32
ikesa_table_segments = 4
init_limit_half_open = 1000
ignore_acquire_ts = yes
syslog {
ike_name = yes
log_level = no
daemon {
app = 1
asn = 1
cfg = 1
chd = 1
dmn = 1
enc = 1
esp = 1
ike = 1
imc = 1
imv = 1
job = 1
knl = 1
lib = 1
mgr = 1
net = 1
pts = 1
tls = 1
tnc = 1
}
}
install_routes = no
plugins {
}
}
include strongswan.opnsense.d/*.conf
@Beleggrodion
Thanks! My colleague found this but it's not related to the behavioural change: https://github.com/opnsense/core/commit/7993a82e84
> FW 24.7.1 with "Automatically generated rules (end of ruleset)"
Which rule is that exactly?
Can you check System: Configuration: History for a revision change of the IPsec migration? This diff would help to understand this further.
Thanks,
Franco
Ok I found it. Will share a fix shortly.
Cheers,
Franco
https://github.com/opnsense/core/commit/178ef826f7
# opnsense-patch 178ef826f7
# /usr/local/opnsense/mvc/script/run_migrations.php
# /usr/local/etc/rc.filter_configure
Cheers,
Franco
And this is probably the actual reason since I read the last post again and Beleggrodion observed it in reverse...
https://github.com/opnsense/core/commit/0e4cb12f3f8
# opnsense-patch 0e4cb12f3f8
# /usr/local/etc/rc.filter_configure
Cheers,
Franco
@franco,
I see the Automatically generated rules (end of ruleset) after applying both patches and running related commands. Testing now on legacy tunnels.
As for non legacy tunnels we do or do not need to add in WAN rules for inbound IPSEC? I have not added any rules for ESP/ISAKMP/NAT-T and my non-legacy tunnels appear to be working correctly.
There is no need to add the same rules twice. That said, I'd recommend to set up proper firewall rules manually and untick that "magic" checkbox. Avoid all similar disruption in the future.
I dont see "magic" rules except for the Tunnel Settings (legacy) site to sites, for the Connections (non legacy) i dont see the same "magic" in Automatically generated rules (end of ruleset).
I dont mind the "magic" as it saves me from having to create a rule for each Site to Site I have, granted I could prob make 1 ruleset and apply to an alias and add all the site to site IPs in the alias. Just easier with the magic and im lazy
Plus im in the process of migrating the last few tunnels I have under legacy to the newer connections.
Quote from: doktornotor on August 26, 2024, 11:23:59 PM
There is no need to add the same rules twice. That said, I'd recommend to set up proper firewall rules manually and untick that "magic" checkbox. Avoid all similar disruption in the future.
Quote from: danderson on August 26, 2024, 11:35:35 PM
I dont see "magic" rules except for the Tunnel Settings (legacy) site to sites
Yes, as already noted - https://docs.opnsense.org/manual/vpnet.html#firewall-rules
With the "magic" gone, nothing works - legacy or not.
Finally reaching out for some help after following this thread and applying both patches Franco released the other day, and I am still struggling with IPSEC tunnels dropping. I am terminating between a Sonicwall 2650 and OPNSense, prior to 24.7.2 no issues, now having issues w/ P2 dropping. Below is the issue I think and I have validated that proposals match:
2024-08-28T10:25:58-05:00 Informational charon 06[IKE] no acceptable proposal found
2024-08-28T10:25:58-05:00 Informational charon 06[CFG] configured proposals: ESP:AES_CBC_256/HMAC_SHA2_256_128/MODP_2048/NO_EXT_SEQ, ESP:AES_GCM_16_256/MODP_2048/NO_EXT_SEQ
2024-08-28T10:25:58-05:00 Informational charon 06[CFG] received proposals: ESP:AES_CBC_256/HMAC_SHA2_256_128/NO_EXT_SEQ
2024-08-28T10:25:58-05:00 Informational charon 06[ENC] parsed CREATE_CHILD_SA request 31 [ SA No TSi TSr ]
If I restart the IPSEC service, the tunnel will come up and stay up for an hour or less and then I see P2 disconnected.
OPNSense WAN FW Rules, see attachments.
Here what I have selected in OPNSense and SonicWall, see attachment.
Quote from: guyp2k on August 28, 2024, 06:27:42 PM
Finally reaching out for some help after following this thread and applying both patches Franco released the other day, and I am still struggling with IPSEC tunnels dropping. I am terminating between a Sonicwall 2650 and OPNSense, prior to 24.7.2 no issues, now having issues w/ P2 dropping. Below is the issue I think and I have validated that proposals match:
2024-08-28T10:25:58-05:00 Informational charon 06[IKE] no acceptable proposal found
2024-08-28T10:25:58-05:00 Informational charon 06[CFG] configured proposals: ESP:AES_CBC_256/HMAC_SHA2_256_128/MODP_2048/NO_EXT_SEQ, ESP:AES_GCM_16_256/MODP_2048/NO_EXT_SEQ
2024-08-28T10:25:58-05:00 Informational charon 06[CFG] received proposals: ESP:AES_CBC_256/HMAC_SHA2_256_128/NO_EXT_SEQ
2024-08-28T10:25:58-05:00 Informational charon 06[ENC] parsed CREATE_CHILD_SA request 31 [ SA No TSi TSr ]
The Sonicwall does not have PFS enabled. This is confirmed by the missing
MODP_2048 in received proposals. You should pick "default" for your ESP.
FYI: Shipping these to patches in 24.7.3 today.
Quote from: allan on August 29, 2024, 01:49:43 AM
Quote from: guyp2k on August 28, 2024, 06:27:42 PM
Finally reaching out for some help after following this thread and applying both patches Franco released the other day, and I am still struggling with IPSEC tunnels dropping. I am terminating between a Sonicwall 2650 and OPNSense, prior to 24.7.2 no issues, now having issues w/ P2 dropping. Below is the issue I think and I have validated that proposals match:
2024-08-28T10:25:58-05:00 Informational charon 06[IKE] no acceptable proposal found
2024-08-28T10:25:58-05:00 Informational charon 06[CFG] configured proposals: ESP:AES_CBC_256/HMAC_SHA2_256_128/MODP_2048/NO_EXT_SEQ, ESP:AES_GCM_16_256/MODP_2048/NO_EXT_SEQ
2024-08-28T10:25:58-05:00 Informational charon 06[CFG] received proposals: ESP:AES_CBC_256/HMAC_SHA2_256_128/NO_EXT_SEQ
2024-08-28T10:25:58-05:00 Informational charon 06[ENC] parsed CREATE_CHILD_SA request 31 [ SA No TSi TSr ]
The Sonicwall does not have PFS enabled. This is confirmed by the missing MODP_2048 in received proposals. You should pick "default" for your ESP.
Thank You, I enabled PFS on the SW and all is good.
Hello my friends,
I was still having big problems with our IPsec site-to-site setup (especially with the two locations using PPPoE DSL). The solution that worked for me was the following:
"The host suggested setting the MSS to 1300 for IPsec connections. I did this under Firewall -> Settings -> Normalization -> Max MSS 1300 for the IPsec interface. To test if this setting works, I tried pinging over the tunnel with a payload larger than 1300 and the 'Don't Fragment' flag."
We didn't have this issue before @franco – maybe you can take a look at it, and perhaps my solution will help others.
Cheers,
Ruxor