Topics

1

General Discussion / Migration of network structure

« on: November 27, 2024, 06:05:34 pm »

For historical reasons, I have the following network structure:

• Fiber ONT (Deutsche Glasfaser) <-> Fritzbox 7490 (router mode) <-> Opnsense <-> …

The above scenario worked fine for the last year (without interruptions). But the signal reception of my DECT phones connected to the Fritzbox is not good in some parts of my house, so I want to move the Fritzbox to a new location. This is a good time to remove the Fritzbox from the WAN side of the Opnsense and put it as a dedicated device in a separate VOIP VLAN. So far, so good. A few questions arise.

• The network card in the WAN (Intel X553) had repeated connection losses in the past, which I solved by disabling EEE (Energy Efficient Ethernet) on the Fritzbox. I am not sure if the Opnsense network card (Intel X553) supports configuring EEE itself (setting the system tunable "dev.ix.n.eee_state=0" via SSH results in a hung SSH session. Finally, this parameter is not set). Does anyone have recommendations to avoid such connection loss issues in advance?
  
  
• There were some discussions in the forum in the past about missing IPV6 prefix and address assignments (especially in case of connection loss) with Deutsche Glasfaser. On my Fritzbox, such problems never occurred in the past. Does anyone know the reasons some users have pointed to? I think the DHCP DUID should be persistent at least during boot cycles. Is that correct?
  
  
• Modern SIP clients should derive the IP address for the SIP server by querying the SRV DNS record instead of directly querying the A record. Does the Opnsense firewall support DNS-based firewall rules based on SRV records?
  

2

24.1 Legacy Series / Wireguard: IPv6 within tunnel broken since update to 24.1.3_1

« on: March 17, 2024, 02:04:32 pm »

Since upgrading to Opnsense 24.1.3_1 (from 24.1.2_1) IPv6 within the wireguard tunnel is not working anymore until restarting the service.

Configuration:

• Roadwarrior connection from Android to Opnsense via IPv6
• Static private IPv4 and IPv6 (ULA) address on the Smartphone
• Routing IPv4 and IPv6 internet traffic through the tunnel

Everything worked fine till the upgrade to the above version. After upgrading and performing reboot of the Opnsense, IPv6 traffic in the tunnel is not routed anymore. A packet dump shows that the traffic still arrives at the virtual wireguard interface (wgX). Thus, wireguard itself (IP addresses and allowed IP addresses) is correctly configured. But the firewall rules do not see any IPv6 packets (no ULA packets are visible in the firewall's live view even logging of all packets is enabled).

I did some checks:

• The firewall rule configuration look fine (even on console via 'pfctl -s rules')
• Routing table looks fine (even on console via 'netstat -rn')
• Output of 'wg' command on console look fine

The only weird entry in the wireguard I have found is:

Code: [Select]

2024-03-17T12:24:50 Error wireguard /usr/local/opnsense/scripts/Wireguard/wg-service-control.php: The command
   '/usr/sbin/arp -s '10.2.x.x' '00:1d:ec:xx:xx:xx'' returned exit code '1', the output was 'arp: cannot
   intuit interface index and type for 10.2.0.3'

The result of the arp command is clear. The related IP address belongs to an interface which is currently not active. But, I don't have a clue why the wireguard control script tries to create static arp entries on the opnsense (for a client which has no relation to wireguard). In my eyes, this looks like a bug, but possible not causal for the described problem.

Restarting the wireguard service after booting the machine for the specific interface solves the problem.


Does anybody has an idea how to track this down?

3

Hardware and Performance / BUG: Low data throughput (bug in Opnsense or FreeBSD?)

« on: November 15, 2023, 06:04:23 pm »

Since migrating from Opnsense 19.7 to 20.1, data throughput when routing between internal network interfaces has degraded significantly. The primary reason was initially the migration to the new network stack based on "iflib", which also required changes to the network driver code. As I recall, there were initially major problems with the ISR (Interrupt Service Routine) in the Intel network drivers, which led to performance degradation. This problem appears to have now been resolved. In fact, the firewall provides full gigabit throughput between LAN interfaces as long as the "Strongswan" and "Samplicate" services are disabled.

When the "Strongswan" service is activated, throughput drops massively. The same thing happens when the "Samplicate" service is activated.

• Strongswan activated: 115MB/s --> approx. 80MB/s
• Samplicate activated: 115MB/s --> approx. 65-75MB/s
• Strongswan and Samplicate activated: 115MB/s --> approx. 50MB/s

Unfortunately, the low data throughput problem has been present since version 20.1 and affects all network traffic, regardless of whether it is intended for the IPsec tunnel or not. I could still live with deactivating "Samplicate", but I have to rely on IPsec (Strongswan). The problem doesn't only occur with my firewall (see link).

Detailed investigations regarding IPsec have shown that "Strongswan" (IKE daemon) itself is not responsible for the lack of data throughput. The data throughput drops as soon as there is at least one entry in the IPsec SPD (i.e. IPsec part of the kernel). I wasn't able to do a more detailed analysis of "Samplicate".

Example:
=========

• "Strongswan" activated and "Samplicate" deactivated
• Data throughput approx. 80MB/s
• Deletion of all entries in the Security Association Database (SAD)
    $ setkey -F
• Data throughput approx. 80MB/s
• Additionally, deletion of all entries in the Security Policy Database (SPD)
    $ setkey -FP
• Data throughput approx. 115MB/s
• Create any security policy outside the IP address range of my own network (policy does not apply)
    $ echo 'spdadd 192.168.222.0/24 192.168.223.0/24 any -P in ipsec esp/tunnel/1234-1235/unique:30;' | setkey -c
• Data throughput approx. 80MB/s

I don't know if this bug is in Opnsense itself or in the underlying FreeBSD 13.2. Has anyone here in the forum made similar observations or also had problems with data throughput in this regard? It would be great if the performance problem will finally be solved after several years.

• Board: Supermicro a2sdi-4c-hln4f
• Opnsense: 20.7.8_1

4

General Discussion / [solved] Reconfiguring WAN Interface (and staying secure)

« on: August 11, 2023, 12:20:12 pm »

In the past my Opnsense was connected to the Internet over DSL using PPPoE. Now, time has gone and my ISP has changed. From now on, my Internet connection uses a cable router (CGNAT for IPv4 and native IPv6). So, I have to reconfigure the WAN interface by switching from a virtual PPPoE interface to a native ethernet interface (IPv4 over CGNAT and native IPv6). Double/triple NAT doesn't matter here and will be adjusted in the future.

I am not really sure how Opnsense identifies the WAN interface. In the past I used the console, but configuring the WAN interface this way drops the whole network configuration.

I guess that generally all network interfaces are equal, regardless of whether they are used as WAN or LAN. So, I did the following steps for reconfiguration and like to get your comments whether I did everything right from practical Opnsense behaviour regarding to stay secure with my adjusted firewall setup.

• Disabling the PPPoE interface
• Configuring the native ethernet interface (formerly the parent of the PPPoE interface) to DHCP
• Reconfiguring the default gateway (System -> Gateways -> Single) to the new public interface
• Adjusting inbound and outbound NAT to the new public interface
• Moving firewall rules from the PPPoE to the new WAN interface

5

General Discussion / IPsec: UDP packets do not find their way back to the socket

« on: March 13, 2023, 09:10:59 pm »

I run two IPsec S2S tunnels to different locations (another Opnsense instance and a Fritzbox). At each tunnmel endpoint operates a DNS as an authorative DNS for internal domains.

1. Opnsense <------ IPsec (IKEv1) ------> Fritzbox 7490
2. Opnsense <------ IPsec (IKEv2) ------> Opnsense


These DNS are used by the unbound instance of my Opnsense via Domain overide. I noticed that the unbound instance on my Opnsense sporadically has communication issues with the DNS of the Fritzbox. This lead me to start investigation. I reconfigured unbound on my local instance to use TCP for upstream DNS of the Fritzbox instead of UDP (by directly modifying the unbound config). Result was, that DNS communication to the Fritzbox now runs fine. The drawback is that the modification of the unbound config is handcrafted which results in being regularly overwritten by the Opnsense due to Opnsense is not aware of this config parameter.


Summarized:
==========

• TCP and UDP communication to the remote Opnsense (2) over the IPsec tunnel runs fine
• TCP communication to the Fritzbox (1) over the IPsec tunnel runs fine
• UDP communication to the Fritzbox (2) over the IPsec tunnel seems to have issues

I concentrated on the IPsec tunnel number 1 (Fritzbox) and investigated further by manually triggering DNS lookups with the "dig" tool in different scenarios:

• Direct DNS lookups to the Fritzbox using TCP and Opnsense as the endpoint (Opnsense command line):  --> works fine
  
    $ dig +tcp -b 10.2.100.1 @192.168.1.1 fritz.box
  
  
• Direct DNS lookups to the Fritzbox using UDP and Opnsense as the endpoint (Opnsense command line):  --> times out
  
    $ dig +notcp -b 10.2.100.1 @192.168.1.1 fritz.box
  
  
• Direct DNS lookups to the Fritzbox using TCP and UDP and a client computer behind the Opnsense:  --> both work fine
  
    $ dig +tcp -b 10.2.100.123 @192.168.1.1 fritz.box
    $ dig +notcp -b 10.2.100.123 @192.168.1.1 fritz.box
  
  

A packet dump shows that the IPsec tunnel works fine and all packets traversing it are correctly decrypted. Even in the 2 scenario (which times out) the Fritzbox DNS correctly sends a response back to my local Opnsense instance. And packet dumps of UDP DNS responses from scenario 2 and 3 look the same.


The question is, why the DNS responses of scenario 2 do not reach the local socket. The firewall rules cannot be the cause of this issue because DNS request and response packets traverse the IPsec tunnel. Furthermore…

• Every DNS requests creates a related entry in the state table of the firewall
• Response packets have a size of around 200 bytes and are far away from exceeeding the MTU
• The UDP issue only sporadically occurs. There are somne days scenario 2 works fine and some days not

I also was not able to force the one or other result of scenario 2 by restarting the IPsec tunnel, IPsec service or the WAN connection.
This issues is not related to Opnsense 23.x. only. It may have occurred in earlier versions. I don't exactly know.


Does anybody has a clue what's going on?

6

23.1 Legacy Series / [Solved] Update migration of IPsec with "Mutual RSA + EAP-MSCHAPV2" broken.

« on: February 12, 2023, 10:45:32 am »

A few days ago I updated my Opnsense from version 22.7.11_1 to 23.1_6 and noticed that some of my roadwarrior IPsec connections do not work anymore. The username and password for the second authentication round (EAP-MSCHAPv2) is not accepted by the Opnsense.

I investigated the configuration file (/usr/local/etc/swanctl/swanctl.conf) and saw the possible issue. The following shows a config excerpt of the affected connection.


        local-0 {
            id =
            auth = pubkey
            certs = cert-1.crt
        }
        remote-0 {
            id = %any
            auth = pubkey
            eap_id = %any
        }
        remote_addrs = %any
        encap = no
        dpd_delay = 10 s
        dpd_timeout = 60 s
        pools = defaultv4
        remote-1 {
            auth = eap-mschapv2
        }


It looks like the parameter "eap_id" is misplaced in section "remote-0" which only handles the first certificate based authentication. The parameter must move to section "remote-1" which handles the second authentication round based on password based authentication (mschapv2).

A fix would be appreciated. But, if the new configuration interface in the WebUI is stable enough, I can try to fix the issue that way.

• Does anybody already has experience with the stability of the new WebUI interface (Connections [new])?
• What about the old config dialogue, can it still be used in parallel for editing connections and manual tests regarding migration?

7

General Discussion / Prevent DNS-Tunneling

« on: January 30, 2023, 09:42:50 pm »

To prevent data exfiltration from the server network in case of possible compromise I'd like to prevent DNS tunneling for this network. Actually, I use "unbound DNS" as a local resolver. Compared to the local networks the server network only needs a handful of hostnames to resolve.

As far as I know Unbound does not support black/whitelisting on an interface basis. So, I plan to use "Bind" as a filtering DNS forwarder in front of Unbound to filter DNS requests of the server network. Perhaps, Bind can completely replace unbound in the future. But at first, I don't want to replace Unbound.

Before starting, I like to get your ideas for preventing DNS tunneling. Thanks.

8

22.7 Legacy Series / Opnsense 22.7 stuck after upgrade

« on: August 22, 2022, 04:09:50 pm »

Hi all,

yesterday I started to upgrade my Opnsense from 21.1.10 to 22.7.2. The upgrade process to 22.7.0 went fine till reboot. After rebooting the sense never came up and the console showed that booting was stuck at "Configuring dynamic DNS clients..."

I further updated to version 27.7.2, but the same issue still exists.

After the system was stuck I got the console after pressing Ctrl-C, so I could manually update to the latest version and start the text based opnsense-shell. Using the latter, restarting all services did not change anything. I also tried disabling DDNS by modifying the config.xml, but no chance.

Has anybody noticed a similar issue or has an idea how to debug?

9

22.1 Legacy Series / IPsec: Mismatch with multiple roadwarrior profiles

« on: March 06, 2022, 06:17:48 pm »

In the last days I did a lot of investigation regarding my roadwarrior connections. I have four different connection profiles active:

• Mutual RSA + EAP-MSCHAPv2 with IPv4 (used by an android smartphone with the strongswan VPN app)
• Mutual RSA with IPv4 (used by an ubuntu laptop with strongswan and the network manager)
• Mutual RSA with IPv6 (used by an ubuntu laptop with strongswan and the network manager)
• Mutual RSA + EAP-MSCHAPv2 with IPv6 (used by an android smartphone with the strongswan VPN app)

The different profiles are neccessary because for flexibility (internet protocol) and different support by the IPsec clients. I tried to get all profiles to work, but no luck. The android smartphone can successfully authenticate with IPv4 but not IPv6. And the Laptop can instead use IPv6 but not IPv4.

I far as I know Opnsense still allows only to add one roadwarrior (mobile) connection profile. But strongswan itself has not such a limitation. There was a discussion about it in 2018 [1]. Some more investigation offered that the combination of successful and unsuccessful authentication depends on the sequence of profiles in the ipsec.conf configuration file (as noted above).

Related to a specific IP version charon either tries to match an incoming connection to the first configuration entry or to none of them . Corresponding entries in the log file look similar to

• charon selects the wrong profile
  
  • looking for peer configs matching <local ip>[<local id>] ... <remote ip>[<remote id>]
  • selected peer config 'con2'
  • selected peer config 'con2' unacceptable: non-matching authentication done
  • no alternative config found
• charon cannot find a profile match
  
  • looking for peer configs matching <local ip>[<local id>] ... <remote ip>[<remote id>]
  • no matching peer config found

According to the strongswan documention (FAQ - no matching peer config found) [2] charon tries to find the correct profile by comparing the ip addresses and identities (including the type of the identity). I don't know, whether the mismatch is based on the wrong identity type. The FAQ recommends in such cases to check the log file (log level 3). Unfortunately, I cannot find a hint in the log file, which identity type the client has been used.

In my eyes it seems to be a bug in charon (strongswan). Because, in case I only activate the last of the above profiles, the IPv6 based VPN on the smartphone works well even when pinning the identities of the endpoints to their certificate's DN. After adding the IPv6 profile for the laptop (still in the above sequence) the IPv6 based VPN connection of the smartphone fails because charon does not find any matching profile. In the second case I could understand that charon mistakenly selects the wrong profile. But, in this case it cannot find any match (second error description above). That sounds weird.

Does anybody know what I am doing wrong or if there is a really a bug in strongswan (v.5.9.5)?


[1] https://forum.opnsense.org/index.php?topic=9142.msg44734#msg44734
[2] https://wiki.strongswan.org/projects/strongswan/wiki/FAQ

Thanks.

10

22.1 Legacy Series / IPSec usage and security

« on: February 25, 2022, 12:07:06 pm »

Yesterday, I have upgraded my Opnsense from 21.7.8 to 22.1.1_3. The upgrade worked flawlessly (thanks to all the developers and the great community). Unfortunately, many of my mobile IPSec connections do not work anymore. I am still investigating and it looks like strongswan rejects the client certificates because of unknown trustworthy. Perhaps, anybody of the early adopters already has experiences with mobile IPSec connections after upgrading to new Opnsense 22.1.x.

Maybe, the problems have something in common with the security related misconfiguration of strongswan I addressed in the past. This all makes me think to either switch to another VPN technology (e.g. Wireguard) or to drop all automatically generated VPN profiles and add my manual ones (provided they won't get overwritten during configuration changes within Opnsense).

Does anybody have some recommendations?

11

22.1 Legacy Series / [Solved] Strange behaviour with realtek USB NIC

« on: February 25, 2022, 10:50:39 am »

After upgrading Opnsense from 21.7.8 to 22.1.1_3 I have problems with my Realtek USB NIC which is only used for a separate printer LAN. So, performance doesn't matter. I know about the discussions regarding realtek device support by FreeBSD's original drivers. The USB card is a quite old one and worked flawlessly with Opnsnese 21.7.x and should be supported by FreeBSD.

Before upgrading I installed the plugin "os-realtek-re", but I cannot verify which driver is in use and how to switch to another one.

usbconfig shows the following results, so I guess "ure" is the driver in use

Code: [Select]

root@opnsense-host:~ # usbconfig ugen0.2 show_ifdrv
ugen0.2: <Realtek USB 10/100/1000 LAN> at usbus0, cfg=0 md=HOST spd=HIGH (480Mbps) pwr=ON (350mA)
ugen0.2.0: ure0: <Realtek USB 10/100/1000 LAN, class 0/0, rev 2.10/31.00, addr 1>

The strange thing is the NIC stops working after link changes from down to up state (the printer is directly connected via a patch cable). Packet capture shows that packets are probably sent but no ones are received. But, with only one hands-up step I get the NIC to work properly again. Stopping and restarting the interface at the command line is a workaround

Code: [Select]

$ ifconfig ue0 down && ifocnfig ue0 up
Does anybody have a clue on that? Or can anybody tell me how to test the other NIC driver via command line (kldload etc.)?

12

21.7 Legacy Series / Security issues and user experience of IPSEC implementation

« on: August 26, 2021, 03:37:41 pm »

I like to restructure my IPSEC VPN endpoints and noticed some inconsistencies in the configuration options which motivated me to have a deeper look into the IPSEC implementation with strongswan. I observed some security relevant issues but there can be even more. Of cource, strongswan is very flexible in its configuration and it's very difficult to map all of its configuration possibilities to the web gui. Currently, the gui lacks of flexibility in configuring the endpoint authentication, which is one of the most important parts of VPN security besides data confidentiality and integrity.

This thread should discuss alternatives to the current implementation to improve security and flexibility.

Observations regarding my IPSEC connections:

• Site-to-Site with IKEv2: Authentication based on "Mutual RSA"
  
  • "My Certificate Authority" has a confusing description and maps to "rightca" configuration option. The parameter should be named to "Remote endpoint authentication CA"
• Roadwarrior with IKEv2: Authentication based on "Mutual RSA + EAP-MSCHAPv2"
  
  • There is no possibility to configure which remote endpoint certificates are acceptable (neither leaf certificates or certification authorities) and no corresponding "rightcert" or "rightca" configuration options are placed in the ipsec configuration file. I do not know, how this is handled by strongswan. I guess, in this situation all remote endpoint certificates which belongs to any trusted CA are accepted. This could be a big security risk.

Recommendations:

• Adapt the gui to follow the strongswan configuration file in ways that parameters like "leftauth, rightauth, leftcert, rightcert, rightca" etc. are configurable on a per connection basis
• Separate authentication rounds for IKEv2 (xauth for IKEv1 respectively), e.g. "Auth 1: Mutual RSA" and "Auth 2: EAP-MSCHAPv2" instead of "Auth: Mutual RSA + EAP-MSCHAPv2"
• Allow configuring multiple dedicated roadwarrior connections with their own IP pools
• Move from deprecated "ipsec.conf" to "swanctl" (swanctl.conf and strongswan.conf")
• Make strongswan aware of revoked certificates (can be challenging). For now, users probably feel secure in case they revoke certificates of compromised private keys within the trust center.

My Opnsense: v.21.7.1-amd64

Maybe, somebody has additional ideas.

13

Web Proxy Filtering and Caching / Update script for blacklists of the squid proxy buggy

« on: December 02, 2020, 02:12:41 pm »

I encountered problems while updating or adjusting the categories of the proxy blacklists. Currently, several categories of the following blacklists are active:

1. Shallalist (http://www.shallalist.de/Downloads/shallalist.tar.gz)
2. UT1 (https://dsi.ut-capitole.fr/blacklists/download/blacklists.tar.gz)

The python script for updating the above lists consumes 100% of cpu for a very long time (more than 30 minutes) which induced me to do some more investigation. I identified the following issues:

• The update process is not transactional. Multiple instances of the update script can be launched the same time which results in conflicts
• During the update process blacklists (files) are rebuilt while in use by squid :-(. I do not know whether this affects the running squid instance with its open file descriptors. But, interruption of the update process (e.g. restarting the opnsense) leaves the blacklists in an inconsistent state which prevents restarting the squid proxy (see following error message).
• Update script contains several off-by-one errors in comparison instructions (e.g. if (len(self._url) > 8 and self._url[-7:] == '.tar.gz')

Code: [Select]

Nov  8 14:11:09 opnsense-host squid[37972]: FATAL: Bungled /usr/local/etc/squid/squid.conf line 32: acl remoteblacklist_UT2 dstdomain "/usr/local/etc/squid/acl/UT2"


14

20.1 Legacy Series / Rapid shutdown on ups shutdown event

« on: September 13, 2020, 12:55:16 pm »

I am using the "nut" package for remote monitoring my ups connected to the server. In case of a power failure and low battery the server immediately sends the appropriate command the opnsense to invoke the shutdown process. This works fine. But the shutdown process of the opnsense takes more than 2,5 minutes.

Is there a possibility to speed up the shutdown process in future versions of opnsense or implement a "rapid shutdown" option for a low latency gracefully shutdown?

15

General Discussion / Squid proxy IPv4 fallback

« on: May 04, 2020, 02:26:56 pm »

Hi everybody,

besides filtering I use the squid proxy in opnsense for IP address translation (LAN: IPv4 only --> WAN: IPv4/IPv6). In general squid prefers IPv6 over IPv4 which works fine, so far. So, in case the DNS resolves an IPv4 and IPv6 address for accessing a server squid tries to use the IPv6 address. But, when the server is not responding squid automatically performs a fallback to IPv4.

In its standard configuration the timeout for fallback is 60 seconds. This is too long because the firefox (and maybe some other browsers) have a shorter request timeout. Thus, the fallback will never occur, provided that the timout in squid is configured with a lower value.

By adding the following line to the squid configuration, the fallback works fine:

Code: [Select]

connect_timeout 7 seconds

Maybe, you can add a configuration field to override the standard timeout?

Thanks.