Messages

I only assumed that the reset code was added later to Linux because it was missing in the FreeBSD driver, which otherwise looked like a port. I really did not do any software archeology. :-)

If that problem is handled in iflib in FreeBSD, it is not really needed in the hardware-specific driver.

If Intel knew that problem existed from the get-go, which would be implied if the hangup fix was there from the first commit, they should have better fixed it in hardware, though.

When I had the problem on my Minisforum MS-01, it did not resolve by a NIC firmware upgrade. Not even disabling global ASPM helped (it did on some china boxes I have).

The problem went away after a BIOS update from Minisforum. So I figure that there are conditions where micro-sleeps trigger a hangup. ASPM potentially allows for those sleep states. It might be the case that Intel also tried to fix it in NIC firmware, however, there are several chip revisions out for I226V and some may have a hardware idiosyncrasy that causes this which cannot be healed via NIC firmware.

Intel seems to have "fixed" the hang problem by detecting it in the Linux driver and issuing a reset - FreeBSD has only now picked up that change in the OS driver (that was what my bug report was about).

So:

1. There are chip revisions (e.g., the latest ones have a smaller structure width and consume less power). You can see that via " pciconf -lv | grep -B3 igc", look at the "rev" value (mine is 0x04 on my current china box, I have others with 0x03). I think that early versions had a hardware problem that cause hangups. I did not investigate up to what revision these problems exist. I also do not remember which revision was on my Minisforum MS-01.

2. On some boxes, disabling ASPM globally via BIOS settings or tuneable does fix the problem, so you should try this.

3. On some boxes, you can get BIOS upgrades (e.g. Protectli, Minisforum).

4. My experience is that if the problems persist, updating the NIC firmare does not help.

5. In the future, we will have a FreeBSD driver with Intel's workaround that handles the remaining problems, the fixed kernel can be tested right now.

Not happening here, neither from a connected client nor from OpnSense itself. So this must be specific to some setups.

Do you have altered MTUs, like with PPPoE or do you use MSS clamping?

Since you deinstalled Zenarmor, did you first enable some kind of hardware offloading, like LRO, to improve performance and forgot to disable it again?

If the default deny rule catches traffic, it is because nothing else allowed the traffic. For TCP, that could be out-of-state traffic that is not caught by an existing rule, which is most often the cause somebody wonders why that rule does not apply.

However, this is UDP traffic, which is stateless, so I figure you have no rule to allow that traffic. For convenience, there is a default "allow any" rule created for the first LAN and for that (V)LAN only. Thus, if you do not allow DNS traffic to your firewall on any other (V)LAN, it will be caught by the default deny rule and thus be blocked.

You may not have noticed yet, because many browsers do not even use DNS on port 53 any more (doing DoT or DoH instead), such that DNS worked despite port 52 being blocked. Maybe you have a rule allowing internet access for your second VLAN, such that 8.8.8.8 can be reached.

P.S.: You created the "noise" all by yourself, because you enabled logging for the default deny rule. The whole purpose of the default deny rule is that you can log what it does - otherwise, you would not need it. While enabling logging on that can be helpful to diagnose something going wrong, blocking packets is the whole purpose of a firewall. Normally, you don't watch it doing its job.

Yes, of course. That is because the delegated prefix (IA_PD) is not within the 'lan network'.

Delegating IPv6 prefixes is an "advanced" topic. The "default allow" rules for the first LAN are legacy convenience rules designed for the standard case only. Also, these rules exist ONLY for the first (V)LAN. You have to create suited rules for any additional (V)LAN by yourself - likewise when you create networks behind your LAN.

As @Franco recommended, you should absolutely try "opnsense-update -zkr 26.1.11-igc_aspm", because that should fix the hangup condition.

BTW: This is the original bug report.

This has nothing to do with Proxmox.

If you want Unbound to resolve local hostnames, you will most likely have the domain osx.ninja forwarded to Dnsmasq on port 53053.
You must do the same for "1.168.192.in-addr.arpa" to enable reverse lookups.

The rules look "O.K." now. I told you to first check if your rules cause the problems in my first answer, for somehow I guessed that you redirected "all" port 53 traffic, creating an endless loop like explained here.

What can you learn of this? Your rule of thumb should be: If you experience problems, show your rules, because often times, they are the cause of it. See also the "READ THIS FIRST" article in the tutorial section.

That being said, your current rules alone will not help you with either DoT or DoH, which are the default in many browsers now.
There is a discussion about this also in the tutorial section.

Basically, you can block port 853 for DoT, but you need a blocklist for known DoH services because you cannot block port 443.

Also, there are a few more kinks in your rules, because they apply to IPv6 as well, see this.

On a side note: I have given up on the "block any other DNS than my own" game, because you cannot win it.

Maybe there will eventually be a fix for this: https://github.com/freebsd/freebsd-src/pull/2318

I proposed to ,,try" disabling These rules in the absence of you having presented your exact rules, which in Turn prevents anyone from inspecting them for any potential errors.

If Unbound works without them being enabled, it would prove something is wrong and then we can start looking. So basically, this is just a quick and Dirty Test, not a permanent Solution.

Both could be blocking Unbound Traffic when slightly wrong.

Try disabling the firewall rules for a try. It might be the case that you created a loop where Unbound tries to query itself when it makes upstream queries.

Why not using good, old IPv4 for internal things?

I am also fond of that, see: https://forum.opnsense.org/index.php?topic=45822.0 for an in-depth discussion.

Correct.

I mean that it is probably the CPU waiting for I/O operations to finish because the storage subsystem is struggling. With NVMe storage, writing very small, frequent log lines leads to heavy 'write amplification' due to the large underlying flash block sizes, causing significant latency spikes.

The fact that reducing the logging activity resolved the issue completely confirms the storage bottleneck theory. When the logging was set to high, the frequent, small writes to the NVMe caused massive write amplification and continuous synchronous I/O blocks.

In FreeBSD/OPNsense, such a severe storage backlog causes 'Lock Contention' in the kernel and forces an influx of Software Interrupts (SWIs) to manage the blocked I/O buffers. This is why the IRQ handler threads (intr) ran hot: they weren't just processing network packets, but handling the massive overhead of a stalling filesystem trying to commit log rows to a struggling SSD. Turning down the logs broke this vicious cycle.