Messages

Maybe there will eventually be a fix for this: https://github.com/freebsd/freebsd-src/pull/2318

I proposed to ,,try" disabling These rules in the absence of you having presented your exact rules, which in Turn prevents anyone from inspecting them for any potential errors.

If Unbound works without them being enabled, it would prove something is wrong and then we can start looking. So basically, this is just a quick and Dirty Test, not a permanent Solution.

Both could be blocking Unbound Traffic when slightly wrong.

Try disabling the firewall rules for a try. It might be the case that you created a loop where Unbound tries to query itself when it makes upstream queries.

Why not using good, old IPv4 for internal things?

I am also fond of that, see: https://forum.opnsense.org/index.php?topic=45822.0 for an in-depth discussion.

Correct.

I mean that it is probably the CPU waiting for I/O operations to finish because the storage subsystem is struggling. With NVMe storage, writing very small, frequent log lines leads to heavy 'write amplification' due to the large underlying flash block sizes, causing significant latency spikes.

The fact that reducing the logging activity resolved the issue completely confirms the storage bottleneck theory. When the logging was set to high, the frequent, small writes to the NVMe caused massive write amplification and continuous synchronous I/O blocks.

In FreeBSD/OPNsense, such a severe storage backlog causes 'Lock Contention' in the kernel and forces an influx of Software Interrupts (SWIs) to manage the blocked I/O buffers. This is why the IRQ handler threads (intr) ran hot: they weren't just processing network packets, but handling the massive overhead of a stalling filesystem trying to commit log rows to a struggling SSD. Turning down the logs broke this vicious cycle.

I would guess this really is more of a storage limitation because of RMW-cycles induced by ZFS.

That CPU is not the fastest, either. For each RMW cycle, it must calculate ZFS checksums. A proper centralized log server could probably buffer log messages more efficiently. If you have Monit running on top, each new message is also parsed and compared against triggers. You could use top and see which processes actually peg the CPU.

It only works for one specific monitoring IP, yes, not multiple destinations per gateway. There is no dynamic routing depending on "what is best".

On the other hand, that would also break your connections, because the routes (and IPs) would change. Normally, you would want "Failover States" to be enabled, too. If you want some destinations to use another (group of) interface(s), you can do PBR based on that.

You can use gateway groups for that and use specific trigger levels - it is in the official docs.

A system like that can be had from Quotom (Q20322G9) with passive cooling, plus it has 4x SFP+ ports on top of 5x 2.5 GbE ports instaed of 5x 1 GbE.

Das ist nicht wahr: Ich sehe "Hetzner DNS" und "Hetzner DNS Legacy (deprecated)" als Service in der GUI angeboten (26.1.11_5).

Do you guys use os-realtek-re or the native FreeBSD driver? You know that Realtek adapters are buggy under FreeBSD, don't you?

Where do you want to migrate the destination NAT rules to?

The only reason to migrate "Outbound NAT" to "SNAT" (aka "source NAT") was that this is the more common term. "Destination NAT" is already called correctly.

[not]

AFAIK, MacOS does requests in parallel, Windows does a fallback and goes back to preferred order. Unbound uses a sophisticated model preferring the fastest server - which is what many clients also do via the "Happy Eyeballs" approach. Whatever the case, this scheme does stop eventually once you factor in IPv6: Because you can only supply DNS server IPs of "your own kind" in any type of IP protocol, you have independent settings. So, which one will be preferred in the presence of both IPv4 and IPv6 DNS servers?

Before you say: "IPv6 is always preferred" - just today I had a case where I had augmented a DynDNS entry for a CG-NAT connection with a fixed IPv4, which has a reverse proxy to make the IPv6-only connection work for IPv4, too. Thus, the DynDNS name now refers to both an IPv4 and an IPv6.

And guess what? That broke a Wireguard PC client, who just happens to prefer IPv4 and thus did not reach the real IPv6 backend, but the IPv4 proxy (which did not have a Wireguard service). So there,

What can we learn from this? Patrick is right: Do not rely on DNS server ordering for ANY client. RFC 2132 is not helpful, either, because it only covers what an IPv4 server SHOULD do - i.e., it does not say:

a. what the server MUST do.
b. what the client SHOULD or MUST do.
c. what the client SHOULD or MUST do when an additional IPv6 DNS server list is provided.