Messages

No, there is a single WAN gateway and a single gateway to reach these routed subnets on TNET.

Scenario: on TNET interface we have a L3 routed network, one example network is 10.10.45.0/24. Routes to these networks are configured in System: Routes: Configuration.

Outbound NAT policies for these networks have been created manually, I assume they're getting hits because at the far end I can see my test traffic originating from the expected IP.

Here's the problem - the replies are being dropped because of "Default deny / state violation rule". My guess here is that the firewall knows the matching state for these rules, otherwise it wouldn't know the private IP they're for. So what is the real issue?
Devices in the TNET subnet work fine, it's just the routed subnets that don't work.

Code Select Expand

TNET        2025-06-17T08:42:25    91.2.119.28:80    10.10.45.49:32975    tcp    Default deny / state violation rule   
TNET        2025-06-17T08:42:17    91.2.119.28:80    10.10.45.49:32975    tcp    Default deny / state violation rule   
TNET        2025-06-17T08:42:09    91.2.119.28:80    10.10.45.49:32975    tcp    Default deny / state violation rule   
TNET        2025-06-17T08:42:05    91.2.119.28:80    10.10.45.49:32975    tcp    Default deny / state violation rule   
TNET        2025-06-17T08:42:01    91.2.119.28:80    10.10.45.49:32975    tcp    Default deny / state violation rule   
TNET        2025-06-17T08:41:59    91.2.119.28:80    10.10.45.49:32975    tcp    Default deny / state violation rule   
TNET        2025-06-17T08:41:57    91.2.119.28:80    10.10.45.49:32975    tcp    Default deny / state violation rule   
TNET        2025-06-17T08:41:56    91.2.119.28:80    10.10.45.49:32975    tcp    Default deny / state violation rule   
TNET        2025-06-17T08:41:55    91.2.119.28:80    10.10.45.49:32975    tcp    Default deny / state violation rule   
TNET        2025-06-17T08:41:54    91.2.119.28:80    10.10.45.49:32975    tcp    Default deny / state violation rule   
WAN        2025-06-17T08:41:54    45.76.130.220:14043    91.2.119.28:80    tcp    let out anything from firewall host itself (force gw)   

I turned the log level back up to DEBUG4 but it's no longer logging these ipdr messages. So I will leave this one for now.

Bug reported.
I notice that in the current version of Zenarmor [vs whatever version it was in July 2024], it doesn't attempt to redirect output at all.

This still happens with current versions of OpnSense + Zenarmor. Simply redirecting the output of "zenarmor peridiocals" to /dev/null would fix this.

Three times in past 4 weeks, disk has filled up on my OpnSense and every time it has been something to do with Zenarmor.
First one was cron jobs that fail to send an error message once per minute. Fixed that.
Second one was mongodb. Sort of fixed that, but haven't actually managed to get the DB engine working again [but at least it's not eating the disk].
This time it is /usr/local/zenarmor/log/active/worker0 logging at about 1MBps. It's mostly lines like this:

2025-03-10T10:01:53.378794 WARN ArrayStream was full, str: osver pos: 8191
2025-03-10T10:01:53.378802 WARN ArrayStream was full, str: ":" pos: 8191
2025-03-10T10:01:53.378811 WARN ArrayStream was full, str: "} pos: 8191
2025-03-10T10:01:53.378820 WARN ArrayStream was full, str: ," pos: 8191
2025-03-10T10:01:53.378828 WARN ArrayStream was full, str: remote_device pos: 8191
2025-03-10T10:01:53.378847 WARN ArrayStream was full, str: ":" pos: 8191
2025-03-10T10:01:53.378855 WARN ArrayStream was full, str: " pos: 8191
2025-03-10T10:01:53.378864 WARN ArrayStream was full, str: ," pos: 8191
2025-03-10T10:01:53.378871 WARN ArrayStream was full, str: community_id pos: 8191
2025-03-10T10:01:53.378879 WARN ArrayStream was full, str: ":" pos: 8191
2025-03-10T10:01:53.378892 WARN ArrayStream was full, str: 1:NLGH3mmPeENTT1aXwYWl5XLicUw= pos: 8191
2025-03-10T10:01:53.378908 WARN ArrayStream was full, str: " pos: 8191
2025-03-10T10:01:53.378916 WARN ArrayStream was full, str: ," pos: 8191
2025-03-10T10:01:53.378924 WARN ArrayStream was full, str: handshake_result pos: 8191
2025-03-10T10:01:53.378931 WARN ArrayStream was full, str: ":" pos: 8191
2025-03-10T10:01:53.378939 WARN ArrayStream was full, str: None pos: 8191
2025-03-10T10:01:53.378964 WARN ArrayStream was full, str: " pos: 8191
2025-03-10T10:01:53.378972 WARN ArrayStream was full, str: }

What is it complaining about here?

In this dialogue we can see that we start on 500 and jump to 4500.
In particular this line: "parsed IKE_SA_INIT response 0 [ SA KE No V V N(NATD_S_IP) N(NATD_D_IP) N(FRAG_SUP) V ]" is this the far end telling me that NAT has been detected?  Because the next packet I send is sent to/from 4500. But I am sure no NAT is in use here.

Code Select Expand

14[ENC1] <the uuid|1> generating IKE_SA_INIT request 0 [ SA KE No N(NATD_S_IP) N(NATD_D_IP) N(FRAG_SUP) N(HASH_ALG) N(REDIR_SUP) ]
14[NET1] <the uuid|1> sending packet: from 203.0.113.158[500] to 109.104.97.188[500] (336 bytes)
03[NET2] sending packet: from 203.0.113.158[500] to 198.51.100.188[500]
02[NET2] received packet: from 198.51.100.188[500] to 203.0.113.158[500]
02[NET2] waiting for data on sockets
12[NET1] <the uuid|1> received packet: from 198.51.100.188[500] to 149.106.180.158[500] (446 bytes)
12[ENC1] <the uuid|1> parsed IKE_SA_INIT response 0 [ SA KE No V V N(NATD_S_IP) N(NATD_D_IP) N(FRAG_SUP) V ]
12[IKE1] <the uuid|1> received Cisco Delete Reason vendor ID
12[IKE1] <the uuid|1> received Cisco Copyright (c) 2009 vendor ID
12[IKE1] <the uuid|1> received FRAGMENTATION vendor ID
12[IKE2] <the uuid|1> received FRAGMENTATION_SUPPORTED notify
12[CFG1] <the uuid|1> selected proposal: IKE:AES_CBC_256/HMAC_SHA1_96/PRF_HMAC_SHA1/MODP_1024
12[IKE1] <the uuid|1> local host is behind NAT, sending keep alives
12[IKE2] <the uuid|1> reinitiating already active tasks
12[IKE2] <the uuid|1>   IKE_CERT_PRE task
12[IKE2] <the uuid|1>   IKE_AUTH task
12[IKE1] <the uuid|1> authentication of '203.0.113.158' (myself) with pre-shared key
12[IKE2] <the uuid|1> successfully created shared key MAC
12[IKE0] <the uuid|1> establishing CHILD_SA c942748f-a0ff-403f-8539-5a2fc2ba54f2{2}
12[ENC1] <the uuid|1> generating IKE_AUTH request 1 [ IDi N(INIT_CONTACT) IDr AUTH N(ESP_TFC_PAD_N) SA TSi TSr N(EAP_ONLY) N(MSG_ID_SYN_SUP) ]
12[NET1] <the uuid|1> sending packet: from 203.0.113.158[4500] to 109.104.97.188[4500] (268 bytes)
03[NET2] sending packet: from 203.0.113.158[4500] to 198.51.100.188[4500]
02[NET2] received packet: from 198.51.100.188[4500] to 203.0.113.158[4500]

I had an annoying issue where a tunnel worked for 48h and then it just "gave up" even though there was traffic trying to flow from both sides [other end is a Cisco ASA but I don't manage it so can't say much more than that].
I set "Start action: Trap+Start" on both child SAs and that seems to have helped, has been up for nearly 90h now.

How is

Code Select Expand

local host is behind NAT, sending keep alives
determined? Is it due to what the far-end says ["you are behind NAT"], or is it some other heuristic? I am seeing it in a scenario where there is definitely no NAT at my end ["local host"] and almost certainly not at the far end.

Firewall with 2x WAN interfaces with public IPs. 1 LAN with private IP. Only routes to far end of tunnel are using WAN interfaces.
I see that "local host is behind NAT, sending keep alives" even though there is no NAT involved. How is the firewall determining that NAT is in use?

Are you sure this does not create any race conditions between the GUI config and the overwritten config?

I am not really sure about anything because I don't have Ph1 + Ph2 up and traffic passing yet and I have guessed how to configure this because I find the documentation to be unclear. However, if the "include" feature in Strongswan turns out to have some race condition which results in an indeterminate configuration, then that would be a pretty serious caveat [which should be documented loud and clear], and probably renders the include mechanism unusable in production environments.

Is it always the same proposals after every reload/restart of the service?

I can test that easily enough with, say, 10 restarts of the service.

[always remembering in future]

Isabella, did you manage to have the tunnel fully working?

I wish I could tell you - the other end just keeps sending back NO_PROP and the administrator doesn't seem to know how to look at his logs. But I am confident that my end is using what I think I have configured, as my logs agree with the config I have created. But you just need to bear in mind that I don't 100% know my assertion of how to configure this is true, because I don't have a ph2 up yet :)

Also, in your custom conf file - did you use the connection id the same as in swanctl.conf, or you created full new settings (including p2 children section etc.)? Thanks!

No, not full new settings. Just overriding some elements of what was configured in UI, and then always remembering in future that what the UI says about the configuration may not be true.
The UUIDs need to match at each level.

Code Select Expand


connections {
    <the connection UUID from swanctl.conf> {
        proposals = aes256-sha1-modp1024
        children {
            <the child UUID from swanctl.conf> {
                esp_proposals = aes256-sha1
       
        }
}
}
}



As I happens, I managed to work this out.
Given a generated configuration:

Code Select Expand

# cat /usr/local/etc/swanctl/swanctl.conf
# This file is automatically generated. Do not edit
connections {
    00000000-0000-0000-0000-000000000000 {
        proposals = aes256-sha1-modp2048
        unique = no
        aggressive = no
<snip>



We create an override .conf file containing:

Code Select Expand


connections {
    00000000-0000-0000-0000-000000000000 {
        proposals = aes256-sha1-modp1024
        }
}


restart the service and the log confirms that modp1024 was used, not modp2048. There is NO indication in the config GUI that this has happened.

Unfortunately vpnet.html is not clear on what is required here.
Am I supposed to do the entire VPN tunnel configuration as a new file in /usr/local/etc/strongswan.opnsense.d/? Or am I able to override one or more parameters by dropping a file in there?
The documentation needs some examples of what custom config looks like.

Does this note in the documentation which says "...will not be mainted by the user interface,..." mean that it won't be possible to choose a custom proposal from the UI? If that is true then how would they be applied to an IPsec policy?