Messages

Disable ASPM, maybe?

So, wie es aussieht, ist mindestens einer der beiden NICs ein Realtek, dafür gibt es ein Plugin (os-realtek-re) mit einem verbesserten Treiber. Teilweise haben diese Geräte unabhängig vom eingesetzten NIC mit Sleep States Probleme, was das Knacken erklären könnte. Man kann ASPM per hw.pci.enable_aspm=0 abschalten.

Dazu die üblichen Verdächtigen: RSS einschalten..., siehe hier: https://forum.opnsense.org/index.php?topic=42985.0

1. It seems to be a somewhat hard task and I doubt that Deciso will re-enact the efforts that Netgate has put into it. Maybe Netgate will at some point make their work public, but for the time being, it is a discriminating feature (even from their own CE product).

2. IDK, maybe that would depend on the specific CPU and which algorithm you choose.

Using an aggressively short validity timespan sure helps, however, you should probably try setting "Deprecate Prefix" to on. That way, radvd will announce a zero lifetime upon restart for the old prefix.

Note however, that this has ill effects when a prefix is being reused - your clients will not accept it any more. IPv6 was never designed for dynamic prefixes, even less for failover situations with different prefixes. Thus, you will die on the road you choose.

Probably the only way to go would be to use NAT66 in order to keep local IPv6 ranges constant, but do not bother using ULAs. There are lots of threads about that, take a look around.

The N1x0 would be alright to route and firewall 10 Gbps. For Wireguard, you need CPU acceleration, but AFAIR, under FreeBSD, the optimized Intel libraries are only used under the commercial version of pfSense - and that is a proprietary implementation, see: https://forum.opnsense.org/index.php?topic=38909.0

That implementation is said to be twice as fast as the normal FreeBSD/OPNsense/pfSense CE one. That being said, I doubt that you will reach 10 Gbps speeds with that class of CPU either way. Measurements of WG speed to a local jail on OpnSense showed ~1.3 Gbps on my N100 and even on modern CPUs like an i5-13500 or Ryzen 9 5950X, it will likely hit no more than 8 Gbps.

Also, when you use some kind of fancy intrusion detection, your speeds will decrease as well. Zenarmor is known to to use only one single CPU thread, so even using a high-powered multi-core CPU would come at its limits.

On the other hand, a hefty CPU will come at a much higher cost for 24/7, so I would give up on my plan to be "future-proof" and buy what suits me now.

Gateway <> WAN-IP.

Do you use a ZFS install? Things like that usually happen when there are filesystem inconsistencies created during hard stops of the netflow process. That cannot happen with ZFS.

Do you happen to have an @-sign in your password and try to login via PuTTY? See: https://github.com/opnsense/core/issues/9888

Ich kann mir eigentlich nicht vorstellen, dass das das Problem ist. Auf manchen Boxen gibt es Probleme, wenn ASPM aktiv ist, das sollte man im BIOS abschalten. Was bei Dir passiert, sind offensichtlich TCP restarts. Das kann auch so etwas sein wie SACK, Window Scaling, RFC1323 und den verwendeten Algorithmus (CUBIC, NewReno) usw. Windows hat da etliche Parameter, die man setzen kann.

Es gab da früher mal Optimierungstools: https://www.speedguide.net/downloads.php , deren Anwendung ist aber heute nicht mehr zu empfehlen.

Neuere Tools sind da eher zu empfehlen, wobei Windows 11 25H2 schon wieder neue Parameter nutzt: https://www.golem.de/news/tcp-die-versteckte-netzwerkbremse-in-windows-10-und-11-2302-172043.html

Wenn man die Einstellungen unter Windows allerdings vergurkt hat, muss man das eventuell zurücksetzen, denn eigentlich sind neuere Windows-Varianten da schon ab Werk ganz gut.

Re-visiting this thread, because I again stumbled over something I already knew, but looking at the initial post, this tidbit might not be clear to everybody:

If you are looking for privacy via encrypted DNS of any sort (DoT, DoH) without using a VPN, you may be out of luck:

a. your ISP is still able to monitor which websites you are visiting.
b. your ISP can also block sites based on their URLs despite your DNS being encrpyted.

This does not even rely on "known" IPs of specific websites, but on SNI during initial TLS handshake, see: https://www.youtube.com/watch?v=H-tPCwxdxpY

I.e.: If you want privacy and/or circumvent internet blocking completely, use a VPN. Of course there are other reasons why you may want to redirect DNS requests, but privacy is not it.

Naja, klar. Wenn die MTU kleiner wird, sind auch weniger Nutzdaten möglich. Hier geht es doch darum, festzustellen, ob der Weg zu den betroffenen Sites eine Limitierung aufweist und Deine Seite darauf einzustellen, auch nicht mehr zu schicken, als möglich ist. Eine MTU von 1400 sollte im Internet jeder können, wenn es also auch nicht funktioniert, wenn Deine Clients nur das nutzen, ist die MTU nicht die Ursache.

Are you mixing tagged and untagged VLANs on the same OpnSense interface by any chance?

If you have sufficient IP address room, you will never need that and this will almost always be true, unless you are a huge corporation where IPAM becomes neccessary.

I once worked for Siemens, who in 2000 had 443.000 employees with a corresponding number of IPs (at least 2 per employee, think of phones and PCs) distributed over hundreds of sites. They needed closely controlled IPAM in order to manage their networks.

It was common to add new IP ranges to certain VLANs when needed. Just because there will always be active devices in any network 24/7, you cannot "enlarge" or "re-assign" existing ranges by replacing a /24 with a different /23 subnet. Because of routing issues, you also need to segment your network into larger prefixes for locations, hopefully those are large enough to accomodate the needed total number of IPs.

Instead, you need to augment additional ranges to a certain network which can then be used for additional devices. I.e.: you enlarge the pool by adding non-consecutive ranges. Thus, the whole used IP space becomes heavily fragmented over time.

Normally, that would be a scenario for VLANs and different subnets, not ranges.

When you use static mappings for affected clients, you might as well send a specific DNS server IP with the DHCP option response that is different from the default. So strictly speaking, there does not even have to be a "range" for this purpose.

Falls es an der MTU liegt, dann kann es auch sein, dass die Gegenseite nicht mit TCP-Fragmentierung klarkommt oder dass UDP verwendet wird.

Dein Skript läuft ja vermutlich auf irgendeinem Client, nicht auf OpnSense selbst. Kannst Du dort die MTU kleiner machen? Vielleicht funktioniert ja das MSS-Clamping auf OpnSense nicht korrekt. Tatsächlich weiß ich, dass das ganze MTU/MSS-Handling für IPv6 seit einiger Zeit nicht ganz wie erwartet arbeitet. Und visualizer.coffee hat auch IPv6-Adressen, die ggf. bevorzugt genutzt werden. Falls dort die PMTUD kaputt ist, müssen dann die Pakete schon an der Quelle längenbegrenzt werden. Ich würde sicherheitshalber mal mit 1400 Bytes probieren.