Messages

[...]
I'd also like to change the MAC on all internal parent interfaces as a security measure against applications leaking firewall OUI / manufacturer info to the outside.
[...]

Why do you think changing the MAC address improves security? Of course, the original MAC address reveals the manufacturer of the network card or motherboard, but only to devices on the same Layer 2 network segment. MAC addresses are not transmitted across routers.

For a long time, I used a custom, locally managed MAC address for the WAN interface to avoid problems with my ISP if my hardware changed. This configuration worked perfectly for many years and with several different ISPs, until a few weeks ago.

What happened? My ISP, Deutsche Glasfaser, was conducting maintenance work in our area. During this exact time, the internet connection went down overnight. Absolutely no communication was possible; the remote end simply didn't respond to any of the sent Ethernet frames. It was a tedious process to resolve the issue, especially because the non-technical support provided conflicting and incorrect information :-(.

After a thorough technical investigation, I discovered that the ISP was using similar MAC addresses for its own infrastructure after completing maintenance. Apparently, the ISP allocated the MAC address range, which also included my custom address, and applied an inbound filter for the CPE to prevent MAC address collisions. After updating the MAC address for the WAN interface, everything worked again.

I just updated from Opnsense version 25.7 to 25.7.1 and noticed that the IPsec VPN on my smartphone is no longer working. The IPsec widget in the dashboard shows "no phase 1 configured", while everything looks fine in the configuration section.

On the console, the "swanctl --list-conns" command confirms that no connections are configured. It seems as if Opnsense is forgetting to derive a correct IPsec configuration from its global configuration file.

After switching back to the previous Opnsense version (v.25.7), everything is working fine again. Has anyone observed a similar issue?

In my eyes it's a bug since wireguard does not need a tunnel endpoint address. My wireguard configuration does not have any configured tunnel endpoint. In the past this worked flawlessly.

During my vacation, I used my roadwarrior Wireguard VPN (Android smartphone -> Opnsense) again after a long period of inactivity. I noticed that IPv6 network traffic wasn't being routed to the internet via the tunnel. IPv4 network traffic, however, worked without any issues. I hadn't changed the configuration in ages. There were only a few Opnsense updates.

I'm currently using Opnsense version 25.7. I tried to reproduce the problem with older Opnsense and older kernel versions. The problem with IPv6 network traffic also exists in the oldest ZFS snapshot (v. 25.1.9_2). So the error must have crept in with an earlier Opnsense update, since I hadn't changed the configuration, and IPv6 in the tunnel had definitely worked in the past.

Further analysis gave the following results:

• A packet capture of the Wireguard virtual interface showed that the IPv6 network traffic from the smartphone arrived correctly at the Wireguard virtual interface, meaning it passed through the tunnel correctly.
• However, the firewall logs did not indicate that IPv6 packets were arriving. Therefore, the problem appeared to be that the IPv6 packets were not reaching the firewall.

Further analysis on the console provided more insight:

Code Select Expand

wg2: flags=10080c1<UP,RUNNING,NOARP,MULTICAST,LOWER_UP> metric 0 mtu 1420
description: My_wireguard_interface (opt15)
options=80000<LINKSTATE>
groups: wg wireguard
nd6 options=9<PERFORMNUD,IFDISABLED>

According to "ifconfig", IPv6 was disabled on the Wireguard interface. Consequently, the packets were filtered on the interface before being forwarded to the firewall. Manually removing the "IFDISABLED" flag in the "nd6 options" or manually assigning an IPv6 address to the interface (which automatically removes the flag) re-enables IPv6, and the network traffic is correctly routed to the internet again via the Wireguard tunnel.

This appears to be a bug in Opnsense, causing the interface to be incorrectly configured. If Wireguard is configured with IPv6, the flag must not be set. Accordingly, Opnsense must remove the flag from the interface if at least one of the following conditions is met:

• In the Wireguard instance configuration, the "Tunnel address" field contains at least one IPv6 address.
• The "Allowed IPs" field of at least one associated peer contains at least one IPv6 address.

In case only condition 2 meets (like in my case), the flag is set and IPv6 traffic is filtered. In my eyes this needs to be corrected in future updates.

If you use a dedicated USB stick and copy the config to the first FAT32 partition you should not run into trouble (for details see here). If you like, you can test this by starting the live system in Virtualbox in your desktop environment. Be sure to always have a recovery strategy to turn back to the old installation in case of unforeseen issues

[...]
So we are back to a mystery.

I got Gigabit throughput on that same board with OPNsense virtualised in bhyve and two network interfaces passed through. Couple of months ago was last I checked.

I have no idea what else I can do. I can test the following scenarios with reference to the thread mentioned (link). After that, I'll probably have to contact the FreeBSD community.

• Create an SPD entry for IPv6 instead of IPv4 and measure the throughput on the LAN
• Upgrade the server to IPv6 and compare the data throughput between IPv4 and IPv6

What I don't understand, however, is why others don't seem to have these problems. The board seems to be used frequently.

I read the last post as installation on a Supermicro A2SDi board without a hypervisor.

I know the board can easily achieve gigabit speeds when routing.

Yes, it does. Running a Linux live system with IP routing, maximum throughput around 110MB/s is reached.

So the question is: which services are you running apart from routing, pf and possibly NAT?
[...]

My Opnsense is running mostly the standard services, extended with

• Nut
• Squid Forward Proxy (not involved in performance degradation between client and server)
• UDP Broadcast Relay

Shutting down non-essential services and kernel modules increases performance, but does not bring back maximum throughput. It looks like the problem is still the known old bug (see here).

However, there is one difference between the old installation (v.24.7.12-2) and the new one (v.25.1.2): when deleting all entries in the SPD (IPsec) and shutting down the Netflow aggregator, the maximum throughput came back to about 100MB/s. In the new installation, the throughput only increases from 50MB/s to about 70-75MB/s.

When I boot the Opnsense live system (v.25.1.2), do the minimal network interface configuration (server: native ethernet interface ix3; client: VLAN on ethernet interface ix2) and create a firewall rule to allow SMB connections from the client to the server, the throughput is about 110MB/s. As soon as I create an additional IPsec rule, the throughput drops to about 80MB/s.

I still don't know how to figure this out.

Any particular reason why you use Virtualbox?
[...]

If it is only being used for evaluation, then fine.

Sorry, I didn't express myself clearly in the first post. My Opnsense runs bare metal :

• Board: Supermicro A2SDi-4C-HLN4F
• Memory: 8GB
• Storage: 120GB SSD

Virtualbox was just the environment to test the migration:

• Checking SSD backup for recoverability, in case something goes wrong
• Installation together with configuration restoration

During the test installation in Virtualbox, it turned out that the configuration import does not work properly if I place the configuration on an additional partition of the installation media. As a result, I was able to adapt the installation procedure to reduce the downtime to a minimum.

[Advantages:]

Today, I migrated my Opnsense from version 24.7.12-2 (UFS) to 25.1.2 (ZFS). It is a complete new installation on the previously deleted SSD. The installation went smoothly. The few manual steps before starting the installation were importing the previously saved configuration and a few additional configuration files.

Board: Supermicro A2SDi-4C-HLN4F
RAM: 8GB

Advantages:

• Installation went smoothly
• System starts much faster than the old system

Disadvantages:

• Still poor data transfer rate between different subnets/VLANs, around 50-80 MB/s over a gigabit connection 😞

[Edit:=====]

Based on the topic segmentation fault I plan do do a clean installation with automatically importing the config file. I want to simulate all in virtualbox before to get everything smooth during the real installation:

1. I tried to boot the Opnsense image directly in Virtualbox. But the image seems to be incompatible and it looks like a general problem of Virtual Box not supporting all scenarios and image formats. However, I created an USB stick with the image for booting the VM.

2. I created and additional FAT32 partition on the USB stick (GPT Type: EBD0A0A2-B9E5-4433-87C0-68B6B72699C7). Then I copied the latest unencrypted configuration backup to /conf/config.xml

3. When using the configuration importer during installation it is not possible to import the file. Neither the correct device "da0" or the partition "da0p5" are accepted. Mounting the partition manually in the Opnsense shell works. Does anybody know what is the reason or what kind of devices the importer accept?



Edit:
=====

• It looks like the importer unexpectedly stops, in case it finds a swap partition, or?
• I did the following workaround: I manually copied the latest config to the backup folder and restored a backup within the live system. Afterwards I started the installer. This works.

Edit 2:
=======

• I have a further question: In case I restore a config without all relevant plugins installed yet and install the plugins afterwards. Are the configuration parts of the plugins automatically applied or lost?

Call me paranoid, but I'm currently not trusting an app with 50k downloads and 0 reviews 😉

That's interesting. Of course, there is no guarantee that the software does not contain any malicious content. Your argument of 0 reviews is very weak. I don't know if there have been any (code) reviews of that app. AFAIK the same applies to the reference implementation of the wireguard android app. Furthermore, the code base of the latter is more than 17 month old. So, it's likely that the app contains any vulnerabilities, which is not the same like deliberately integrating malicious content into an app. The code base of "WG Tunnel" looks more recent, but I haven't checked in detail. If there had been any code reviews of the wireguard android app, the community would know that the app stores any credentials of wireguard profiles in clear within the file system. Any app with root privileges is able to retrieve that credentials :-(. Both apps are open source. Hence, you can review the code and build the apps by yourself.

In general it's is up to the public and community to identify malicious code and behavior of open source code. In my eyes open source projects driven by a one-man show are more susceptible to malicious code than projects maintained by a couple of people. A good example was the backdoor in the xz compression lib identified short time ago. If I remember correctly this project had only one maintainer that time and was easily infiltrated by adversaries. Log4j was another one with a critical vulnerability instead of malicious code. But the result regarding attack vectors was likewise.

Do you know what your smartphone is doing in the background apart from android?

Just my 2  cents.

Not really a solution based on OPNsense, and only for iPhone users:

There is an android app with more functions than the reference wireguard app:

• WG Tunnel (com.zaneschepke.wireguardautotunnel)

With this app you can automatically start the wireguard tunnel when leaving trustworthy wifi networks. I haven't tried this function yet, but it looks promising

Thanks. Presumably, the upcoming system upgrade will be a clean install with ZFS. Hopefully, this will increase system availability in case there are any problems with future updates.

[...]
I was doing further searches on this topic and I came across this post from last month where the exact same concern is being pointed out - https://forum.opnsense.org/index.php?topic=44448.msg224885#msg224885

Hey guys I roughly followed your conversation. In regard to the linked thread I did some more investigation over the time. Actually, I have not yet collected all the necessary information to raise a ticket on github and to describe a good solution to be implemented in future versions of Opnsense. But, the solution is rather simple. In the meantime I use a dirty hack. But, the hack will break IPv6 in case the IPv6 prefix change ( using Deutsche Glasfaser the assigned IPv6 prefix is pretty stable).

From my point of view and technical perspective, there is no  difference between deriving a SLAAC address from router advertisement messages or a dedicated IPv6 prefix for the WAN interface based on DHCPv6-PD.

You can do the following for enabling IPv6 privacy extensions (example):

1. Configure IPv6 privacy extensions in tunables (adjust the time values ��according to your personal preferences)
  - net.inet6.ip6.prefer_tempaddr = 1
  - net.inet6.ip6.use_tempaddr = 1
  - net.inet6.ip6.temppltime 3600
  - net.inet6.ip6.tempvltime 604800
 
2. Configure the WAN interface to DHCPv6
  - request only an IPv6 prefix
  - configure a prefix ID
 
3. On the command line: Based on the dedicated prefix for the WAN interface add an additional IPv6 address with "autoconf" flag enabled (e.g. execute the following command)
  - FreeBSD immediately starts generating new IPv6 privacy addresses
  - Maybe, instead of assigning a new IPv6, setting the "autoconf" flag for the already existing one should also work

Code Select Expand

# ifconfig <interface> inet6 <prefix><id>:1111:2222:3333:4444/64 autoconf


No such reports either here or on Github about 25.1.x

Thanks.

You can play it safe, take a snapshot. If not on ZFS and on bare metal this is a perfect opportunity to do a fresh install importing the config on the fly. (Have a copy of the config file nevertheless)

This is the point I've been thinking about for a while. The installation dates back to 2018 and therefore still uses UFS. Snapshots are the most important aspect for me in order to switch to ZFS soon. Before reinstalling, however, I need to list and save the manually created configuration files, otherwise they will be lost.

I'm not that familiar with ZFS. I suspect 8 GB RAM (non ECC) and 120 GB SSD shouldn't be a problem, or?