Messages

Short update: The FreeBSD patch https://reviews.freebsd.org/D33876 was commited today.

More good news: I installed the newly compiled kernel today on my "productive" home firewall, and things turned out well. Dom0 is running Debian 11.5, kernel version is 5.10.0-19-amd64, so already a couple of updates after the last functional version 5.10.0-10-amd64. Network in OPNsense 22.7 DomU is up. No changes were required besides installing the patched kernel on a normal 22.7 system. I didn't do any thorough tests, though, so be cautious, there might still be hidden traps.

Here are the instructions on how to compile OPNsense: https://github.com/opnsense/tools

Here is the link to the kernel patch: https://reviews.freebsd.org/D33876

Note: I enabled

Code Select Expand

options XENHVM
device xenpci in the kernel config, but that's probably not a must.

Here is the instruction on how to install the kernel file:https://docs.opnsense.org/development/how-tos/kernel_debugging.html

If you want to try my 22.7-kernel, send me a PM. Be aware, I am far from knowing precisely what I did. :-)

Of course, it would be nice if the patch could be pulled into an official FreeBSD version and hence OPNsense soon... Anybody out there who could support the review process?

Good news! I just tested my OPNsense build with patched kernel again on my test DomU, and now the network issue is gone! I can ping other networks, Dom0 bridge and DomU interfaces are up. So it seems to be confirmed that the patch solves the network issue, at least in my setup (Debian 11.5 with kernel 5.10.0-19-amd64 as Dom0, OPNsense 22.7 as DomU).

I don't know if my first test was not done properly or if something else (Dom0 kernel?) changed in the meantime...? Anyway, I will do more tests in the next couple of days on my actual firewall DomU (when my familiy is sleeping :-) and report what I learn from these tests. I can also make available my patched 13.1 kernel, if somebody is interested in testing it. Please let me know.

Hi Torsten,

No news from my side. I didn't get around to do more tests with compiling a kernel with the integrated patch yet.

My first successful OPNsense build _should_ have the patch included, but it didn't solve the networking issue. Because I don't have much experience in configuring kernels and bulding, even less so in FreeBSD, I am not sure if I made a mistake while compiling OPNsense, if there is some setting I need to toggle for using the new xenback driver, or if the patch doesn't solve this issue after all. I can of course share the newly created .iso file for testing if anybody is interested.

My next step woudl be to reduce complexity and test if a vanilla FreeBSD DomU has the same or a very similar networking issue. Then I would compile a patched kernel for vanilla FreeBSD to find out if the patch solves the networking issue there. If it does, then I would continue towards finding out why the patch doesn't seem to work in OPNsense. But my time is limited...

Any suggestions are welcome. :-)

I have set up OPNsense as the primary router between my DSL modem and a secondary router (OpenWRT with WiFi AP):

DSL Modem -> OPNsense -> LAN1 -> OpenWRT -> LAN2

OPNSense gets a /58 prefix from the DSL modem (dynamic IP). Prefix delegation of /60 ranges is set up in DHCPv6 section of OPNsense. The OpenWRT WAN section gets an IPv6 address and a /60 prefix.

Internet access via IPv6 in LAN1 works without any notable issues.

Internet access via IPv6 works in LAN2 as long as OpenWRT is set up to work as an IPv6 relay. Clients in LAN2 then get an IPv6 from the same range as LAN1 (e. g. 2001:0db8:0:d7c0::/64). However, I then have trouble connecting via IPv6 to some internal servers with static local ULAs in LAN2.

Therefore, I set up OpenWRT as a DHCPv6 server to distribute IPv6 addresses from the delegated range (e.g. 2001:0db8:0:d7d1::/64): I can access the static local ULAs, but then internet access via IPv6 fails. I can see in the OPNsense firewall that the traffic from the client in LAN2 is being blocked ("Default deny / state violation rule").

I would expect that traffic from delegated prefixes is automatically allowed to pass the firewall. Is this expectation wrong? Or am I doing something wrong? Any hints are welcome. If you need more information on my setup, let me know.

@nabrog87: Unfortunately, I can't help you with your error. But I suspect that it is not the same error that has been discussed in this thread. The error message is different.

There is no GUI that you can access directly on the system without a browser, even when you install the VGA version, AFAIK. You need to access the OPNsense web interface using the specified IP address (default:  https://192.168.1.1/)?

I used the serial image for an installation on pure Xen, and it is working fine (except for a networking issue with current linux kernels, see https://forum.opnsense.org/index.php?topic=28708.msg146175#msg146175). I can access the console via the Xen console (e.g. xl console opnsense), and I can access the GUI via a browser under the specified IP address. Initial network configuration needs to be done via console, as described in https://docs.opnsense.org/manual/install.html. Of course, in case of a VM, the network/bridges need to be prepared accordingly for the web interface to be accessible.

With the review status of the patch https://reviews.freebsd.org/D33876 and no changelog entry for FreeBSD, I am not surprised that 22.7 has the same issue.

In the meantime, I started working on building OPNsense with the patch included in order to find out if that solves our issue. However, with not much experience in building tools and even less experience in FreeBSD, I did not really get far. (I now have a running FreeBSD installation with XTerm, though. Yay. :)) My first succesful build still had the network issue, but I am not entirely sure if I did everything right. Was somebody else more successful than me?

@nabrog87: Did you look into the log file as stated in the error message? However, this looks like something different enough that it might be better to open a new topic.

Huh, that's weird. The only thing that I can think of is the following: While trying different stuff out, I did a

Code Select Expand

#pkg search pkg This listed (among others) pkg-devel-1.17.99.1. As the version number1.17.99.1 was lower than the official pkg one (1.18.something), I simply tried it out with pkg install --force. It didn't work, the error while building OPNsense remained, but maybe this was the intermediate step that allowed me to install the pkg in the way that franco describes?

@franco: That helped a lot, thank you!

For reference: After building as franco describes,

Code Select Expand

# make reinstall failed. However, I manually installed the pkg file:

Code Select Expand

# cd work/pkg
# pkg install --force pkg-1.17.5_1.pkg

Then (in the correct directory, of course)

Code Select Expand

# make DVD completed flawlessly.

Disclaimer: Being a bloody beginner, I am not sure if this is the right way, so be aware.  ;)

I am in the same place, same error on an up-to-date FreeBSD 13.1 system. Any suggestions how to switch to pkg-1.17.5_1 on my build system? I couldn't find a pkg file for manual install. (I am new to FreeBSD, so please be lenient... :-)

EDIT: Or any other suggestion how to succesfully build OPNsense, of course. I might be on the wrong track...

I just had a more thorough look into the FreeBSD review system, and I can confirm my initial assumption that https://reviews.freebsd.org/D33876 is still open. This leads to my second question from above: What can I do to speed up the review process?

IIUC, there have been security related changes in Xen netback driver, which is part of the dom0 kernel. The changes address potential harmful behavior of the netfront driver, which is part of the HVM domU. Previously tolerated behavior now, with new kernels, triggers a stop of the interface.

The only solution (other than downgrading the kernel, which of course should only be a temporary "fix" at most) is to change the behavior of the FreeBSD netfront driver. I am by no means a developer, therefore I am not sure if (a) the fix described in https://reviews.freebsd.org/D33876 does address the issue described in this thread and (b) if it has already been pulled and is e.g. part of FreeBSD 13.1. The fix status "needs review" sounds rather like it, well, needs review before being pulled into FreeBSD.

Hence my two questions to the community:

• Can anybody confirm or reject my hypothesis? How can I find out if the fix has been pulled?
• And, if it turns out my interpretation of this situation is correct, how can the review process be sped up?

Edit: FreeBSD 13.1 changelog does not mention any changes regarding Xen network drivers.

Regarding the kernel 4.19.0-20-amd64 I have no idea why that didn't work. Perhaps the netback changes had already been backported?

There is FreeBSD code related to the above mentioned mailing list exchange which needs review: https://reviews.freebsd.org/D33876